PROTOTYPE

Would you trust a hearing test on your smartphone?*

*This is the most frequently asked question at Mimi's usability tests.

 

SKILLS

interactive prototyping,
testing,
analytics

 

ROLE

experience prototyper,
interaction designer

 

COLLAB

audio researcher,
product owner,
QA


 

THE PROBLEM

Mimi's sound personalization technology makes your audio crisp & clear for your unique hearing. Provided that we can test your hearing ability accurately.

 

QUANTITATIVE INSIGHTS

Users perform rather poorly at the first recorded test, rendering the results unreliable. On the upside, they to get progressively better, proven by the cleaner and cleaner datasets (we use the fancy term improved test–retest reliability).

 

QUALITATIVE INSIGHTS

Usability tests pointed at the same problem: people never trust their first test because they don't feel proficient enough, and end up redoing it to make sure their results are accurate.

 
 

How might we ease users' learning curve so that their hearing data becomes reliable quicker, and they feel confident about their skills to perform the test?

To tackle this hybrid challenge of usability & perceived confidence, I spearheaded an iterative prototyping project to find the right form of test instructions.


 

CONCEPT DEVELOPMENT

Early on, usability tests revealed that the winner concept succeeds in engaging the user more than a block of text would, but it also has the potential to induce cognitive overload.

 
 

“There's so much to do, listening to the beep, the text keeps changing. I have no idea what the colors might mean, maybe caution, to watch out for something?”

 
 

After a few rounds simplification through low and high fidelity prototypes, the design team was ready to present the concept of an interactive tutorial that nudges users to learn the interaction. We were ready to pop the post-release champagne.

Not so fast.

Building real products takes grit and patience to work with unforeseen constraints.
A technical proof-of-concept revealed that the animations were so CPU-heavy on the tech stack that it couldn't play sound any longer.


 

IMPLEMENTATION

A meagre MVP and a tech stack rebuild later, our concept got a facelift, and was ready to try again.

Learnings

  • This concept gives real-time positive and negative feedback visually.

  • Succint verbal instructions are unavoidable.

  • The interaction model is simpler, without different modalities being added to the same UI element.

practice_test_short.gif

 

VALIDATION

What makes this concept complex?

The friendly UI is deceptively simple. The challenge lies in refining the underlying logic when positive/negative feedback is triggered. A traditional analysis of success metrics (high conversion, low drop-off rate) can't directly show the benefits of the feature. Yet, some form of small-scale validation has to happen before wide rollout. Just like the recognition of the problem, the validation of the solution needs to be data-driven: a purely qualitative approach doesn't cut it.

To achieve this, we need to dig a level deeper into the generated raw data to measure if the test-retest reliability improves in various hearing ability cohorts.

A person with near-perfect hearing has short reaction time, shown by the short distance between press and release events.

A person with near-perfect hearing has short reaction time, shown by the short distance between press and release events.

A person with not great hearing has longer reaction time, shown by the increased distance between press and release events.

A person with not great hearing has longer reaction time, shown by the increased distance between press and release events.

Giving real-time feedback during the test shouldn't distract users or falsify the test results, so reaction time isn't a good metric: people with different hearing abilities react differently to sounds, therefore faster reaction time may actually mean that the user isn't actually reacting to the sound.

This plot is from a user who didn't complete the hearing test consistently: in most cases, this is the sign of bad usability.

This plot is from a user who didn't complete the hearing test consistently: in most cases, this is the sign of bad usability.

The goal of the hearing test practice feature is to reduce the amount of erratic data recorded by users. By A/B testing the addition of the feature, we can measure whether the rate of erratic behavior decreases significantly. The pre-launch evaluation phase is still in progress.