What’s going on? (the problem)
The technology team has been migrating clients with 30-40 million subscribers into a new platform and this was a chance to do user research and usability testing with customers before launch. Although the research is late to the game the team felt it was necessary to successfully launch the platform.
My Role
As my first project at Asurion I was tasked to lead, plan a 1 week study, coordinate, facilitate and analyze qualitative data to take back to the teams. This was a ‘oh my gosh’ project because most of the team has not been exposed with user research or UX methods—I was ready for the challenge.
Users affected: ~250 frontline agents (remote and on-location).
20+ million dollar company revenue driver across two clients and additional 20+ million dollar after migrating another client.
500-700+ claims filed daily on the platform.
2 UX Designers on a cross-functional team of 16.
1 week study: 3 days of user interviews, observations and 2 days of usability testing.
Purpose
User mental, behavior and emotional Model: create personas that is driven from qualitative data and utilize it for solution sessions.
Deep empathy and socialize environments: cadence on “feature driven” personas and “socialize” personas to help teams discover new features
Test and Learn: conduct usability testing and create a cadence on future features
Goals
Influence team on findings and testing methods
Users understand the platform once they enter it
Users handle time reduced by 30 seconds
Users quickly negates fraudulent activity in 60 seconds
Users understands the navigation on the Account page
Challenges of being the ‘new guy’
During the first week I was in learning mode. I was analyzing the platform, learning how the team functioned and partnered with the Product Managers involved with the product. I discovered that the team had minimal knowledge of user research. The team was used to saying ‘users’ during meetings but only ran with assumptions and previous experiences— this was an opportunity for me influence the value of design and take a stab on putting names and faces to the customers.
DISCOVERED
• One change on the platform will affect all clients using it.
• Team was not exposed to usability testing reports and findings
• Team was pure back-end development and front-end resources were limited
Let’s be a fly on the wall (3 days)
I had to quickly create bridges internally to schedule, coordinate and fine-tune the logistics of the research plan. My colleague and I divide and conquered—he worked on the prototypes, while I created the observation and usability testing plan and scripts. This was also an opportunity for me to mentor my colleague on user research processes as he had minimal experience on the steps.
FINDINGS
Participants mentioned they are “used” to the current legacy system
Several participants experience miscommunication from in-store sales reps and that they are difficult to work with
All participants are hoping that the new platform reduces their “handle times” with customers
All participants rely on a faster platform so they are able get bonuses
“I care about the company and I care about my customers that I will do anything to fix their problem. I don’t want to look bad specially when the system is slow”
Let’s test the designs (2 days)
My colleague and I conducted 5 rounds of 1 hour usability testing with experience participants. We simulated a ‘happy path’ scenario as we pretended to be the customers on the other side of the phone. This gave a deeper experience for our participants and provided us insights on several components of the interface.
FINDINGS
All participants understood the “Account” page of the platform
Some participants had a difficult time navigating through the “Notes” section of the platform
All participants were blocked from the “Nickname” feature of the platform
3 out of the 5 participants felt like the platform will actually increase their “handle-time” with the customers which makes them lose their bonuses
“Thats right— this is multi-device protection plan so I will need to deregister this device to register her number...this seems very confusing and it seems like I’m doing more steps””
Research Report
Steps taken on the research:
Contextual inquiries
Ethnography
User interviews
Usability testing
Testing Report
Conducted usability testing with 5 participants
Our takeaways and artifacts
After analyzing the data I created personas with two different use cases
Feature Driven Personas
These personas will be taken into sprint and solution meetings to focus on the current themes that our users are currently experiencing
Socializing Personas
These personas are conversation starters and used for curiosity. They are placed in hallways as large format posters to push the users upfront.
These personas data-driven with a hint of “what is this?” They are used to influence teams passing by their daily work day and my hypothesis is that it will start tiny conversations within the team to understand their customers in a deeper outlook.
A day in the life of a frontline agent
An iteration of the current experience that users are currently going through in their daily work day
Results
Leadership is using the report as a reference for sprint planning and solution meetings
The team disabled two features that were blockers for the users
96% of frontline agents successfully finished a claim with a customer on launch day
System Usability Score (SUS) scores decreased 6%, but the testing reports discovered the pain points
On-going user research for Verizon platform launch
Next steps
Socialize findings throughout the teams and leadership
VP of Product found the personas helpful and want to produce them
Ideate and iterate on upcoming platform features
Update and iterate on persona data
Plan and prepare for Verizon user research