Testing usability of a shipping software

Usability Testing

Product Design

Project Overview

Falcon is a web tool used by the Staples store associates to receive customer's packages and ship them with UPS.

Staples is now partnering with other retailer to allow their customers to return online-purchased products at the Staples stores, and Falcon needs to get ready to let associates receive, package, and ship those Returns.

Role & duration

Product designer - 3 months

Project goal

We needed to integrate a solution for associates to receive, process and, ship customer's new type of returns through Falcon. This is how the process would work:

New Returns service flow

Background

Staples already allows customers to drop-off their pre-labeled packages at the stores. This service is called Drop-offs, and associates process them in Falcon as well, like this:

Current Drop-off service flow

The process

6 weeks

Define & research

Business requirements
Competitor & user research
User needs & journey map

4 weeks

Ideate, test & repeat

User flow
Need-to-feature table
Wireframes
Rapid prototypes

2 weeks

Design delivery

Mockups
Prototype
User & system flows

1 weeks

Plan strategy

Success metrics
Next features

Define

Gathering business requirements

So the project started with a kick-off meeting. Our business partner told us what their goal was, and our role in it. They presented a user flow that highlighted the tasks an associate was going to do as well as a table with possible pain points and solutions.

The initial requirements were sometimes vague, which was expected since this type of service was everyone's first time. So I took the proposed flow and draw a storyboard, hoping it could help us all imagine how the associate's experience was going to be and think of more detailed requirements.

The result was a detailed service map that highlighted how all players were going to be part of the experience and each team's responsibilities for the rest of the project.

Research

Time to investigate

This was a total new service for us as well our new partners. To settle down how the new integration was going to work, I needed to investigate what other competitors were doing and what were the user needs.

Competitors

I found Amazon had recently partnered with Kohl’s (a brick and mortar retailer in the US) to let Amazon customers make their returns at the Kohl’s stores. Better yet, many amazon customers had uploaded their experiences returning products on youtube!.

Users

I had a chance to fly to the states to visit the Staples store where Falcon is used. Even though the goal was to design a new feature, I needed to investigate what was the associate's experience when using Falcon.
‍
I identified 2 types of users that had different behaviours and sometimes different responsibilities, but both used Falcon and shipped packages.

Temporary associates

Associates who plan to work at the stores for only a couple of months.

Work as well in Copy & Print area.
Are able to perform common tasks in Falcon.
Relays on training sessions to learn about tasks and how to use Falcon.
Receive "unacceptable" packages from customers.

Experienced associates

These associates have been around for more than a couple of years already.

Usually handle complaints from customers.
Know most of the ins and outs of Falcon.
Help customers packaging their shipping items.
Perform other associates tasks if they are busy.

Research synthesis

Once I completed the research, I mapped my findings so we can all discuss and agree on what were going to be user needs for the new Returns feature.

Current associate's journey when receiving a "Drop-off"

User needs

From seeing associates performing their jobs in the wild, I found that most of them are new employees, and they have a hard time remembering how to do things either because they learn them during the training or because they got interrupted while doing a task.
I realized that associates needed to do tasks fast and well without having to worry if they forgot how to do them or if they made any mistakes.

Guidance

Be able to find what to do before, during and after the process.

Fast process

A quick process that doesn't hold them when receiving a package.

Trust

A system that prevents them from making mistakes, or undo them if needed.

Design challenge!

And now that I discover the user needs, I knew what Falcon had to do for them. So the challenge was...

How might associates process a Return fast and safe using Falcon?

Ideation

User journey

Now that we knew what the user and business needs were, we all mapped the phases an associate had to go through when accepting a Return from a customer.

New associate's journey when receiving a Return

Possible pain points

New associates might not even know it is possible to receive returns now.
Associates won't know if the customer is bringing a Return or a Drop-off.
Processing a Return will take much longer than processing a drop-off.
Part of the process is in Falcon and out of it. Associate's experience might be interrupted.

TRANSLATE INSIGHTS TO FEATURES

Organizing ideas into features

It was a lot to digest already, I had a possible user journey along with pain points for the new experience, but I still needed to make sense out of those ideas and prioritize the features that we needed to create. So I mapped it all in a table, scenarios, user needs, knowns and questions and of course, Falcon needs.

Table with scenarios, user needs and possible features.

Initial ideas

First iteration

So what is the problem?

Or at least what I thought was the problem initially, was that associates were going to have a hard time initiating a return. How could they know if it was one of those new Returns or a classic Drop-off?

Initial hypothesis

Associates are familiar with Drop-offs already, so if we merge them with the Returns they don't need to decide what type of service to choose. After they initiate a Return correctly, we can constrain them or show instructions to continue to the next step in the flow.

If we group Returns and Drop-offs, then associates can initiate both the same way through the same entry point.

Initial idea

In this idea, we were levering the Drop-offs tab (currently part of Falcon) to let associates enter Return codes as well.
‍
After entered, Falcon would tell the associate if it was a Drop-off or a Return, and once processed, it would instruct the associate how to package it.

Receiving a Return and packaging instructions

Once a Return is inside a box, the associate will have to print and stick a shipping label on the box and ship it once the UPS driver arrives to pick it up.

Creating a Shipping Label flow

Tests

Time to validate assumptions

I wrote down the questions and assumptions we all have made when creating the initial solutions, and then, I put together a test plan which clarified what to do next.

Instead of staples associates, I tested my co-workers, and for the first phase of the project, an inVision prototype along with boxes was good enough, hoping the testers could experience an employee's day at work.

Test results

Not a single tester could even complete the 3 first tasks. They didn't notice Returns were possible, and if they did, they didn't know how to process them.

It was fair for me to say that the initial hypothesis was wrong. It was false to assume that by grouping Returns and Drop-offs, testers would process them correctly.

Nothing to worry about, cause I was going to keep iterating until the solid rock was found.

Result

And on the fifth iteration...

After many updates and tests, the testers were finally completing most of the tasks to process a Return correctly.

The problem changed

Users think Returns and Drop-offs are a similar service. That is why they didn't read any of the instructions in the previous versions, and couldn't complete any of the tasks.

So the hypothesis changed too

I found that by providing Returns its own tailored flow, separated from Drop-offs, users will understand it is a service that should be processed differently than anything else.

Solutions

Returns quick tab

A specific tab for Returns along with instructions for how to accept it, help associates understand they are not processing a Drop-off, and the "New" badge explains that is not something they might have missed during training.

Also, the visual cues on each instruction draw the tester's attention and now they were able to process it correctly. Breaking instructions per phase helped them to focus on each task at a time.

Label the return box

Falcon will now determine how many labels the associate needs in order to prevent any errors. The validations were improved for the associates to only answer questions so Falcon makes the final decision.
‍
Also, in case users forget to generate a shipping label for the returns, Falcon will pop up a reminder every week, as long as there is at least 1 return in the database.

Received returns page

Based on test results, it will be much easier for users to understand the concept of a batch of returns when all the Returns are shown with all the batches.
‍
This way is easy for them to notice if a Return is in the wrong batch and move it to the right one.

Moving Returns

Associates will be susceptible to put items in the wrong batch due to the many tasks they have to perform in the store.
‍
And the same did testers, but eventually realized these grouping mistakes, and tried to solve it by transferring items from one batch to another, so we designed this moving functionality that allows them to organize and fix those mistakes in case they happen.

Final Results

All testers were aware of the new type of service.
All the testers were able to accept and package the returns properly.
65% of the testers were able to create a shipping label for the return box.
35% of the testers were able to move accepted returns across batches.

Compared with the initial results, the final ones were incredibly better. At last, the testers were able to process a return.

What's next?

Time was over for me to design. Now I had to wrap it up and pass it to development. Still, we all had to continue working on the next tasks.

Plan the "nice to have" features.
Continue working along with development to implement the solution in Falcon.
Run a pilot test at Boston's stores.
Measure the integration results and validate the assumptions we had previously made.

Challenges

It was one of the projects I have enjoyed the most. These are some of the challenges I had, mostly due to COVID-19.

Prototypes

Initially, the inVision prototype was good enough to test, but eventually, I needed to test more complex functionality, so I required a more robust prototype. I had to learn Axure from scratch and created an almost fully functional prototype to test in just a couple of days

Testers

Part of the test was for me to bring boxes to pretend to be a customer so the tester could immerse as much as possible in a real associate's experience, all this effort had amazing results (since no one could perform any of the tasks initially 😂). But halfway in the project, COVID-19 attacked and we all had to work from home so I couldn't run the tests with my coworkers or bring boxes.

I tried usertesting.com, so I ran a remote pilot test with a co-worker, and tweaked the test plan to be perfect for the testing sessions, which ended up being better than I thought they would be.

Learnings

Get to the test as soon as possible. Initially, I spent quite some time creating a perfect solution, just to find out it was far from it.
I know that the larger a company, the more rules there are to follow, and I was a bit hesitant about contacting other stakeholders without the approval of my manager and that cost me a lot of time initially. However, I learned to make the best out of the meetings we had and not to hesitate if I needed to contact someone.
Associates don't read, and that is fair, considering all the tasks and the customers they have to deal with at once.
Gathering business requirements with clients was a fun experience. I was used to never challenge them, but I realized that if necessary, there was nothing wrong to question and even propose to tweak them. This project made me see how much the business ends up transforming the user experience, but it also taught me how to influence the business needs to have a better user experience. One can't live without the other.