Would you trust a hearing test on your smartphone?*
*This is the most frequently asked question at Mimi's usability tests.
Mimi's sound personalization technology makes your audio crisp & clear for your unique hearing. Provided that we can test your hearing ability accurately.
Users perform rather poorly at the first recorded test, rendering the results unreliable. On the upside, they to get progressively better, proven by the cleaner and cleaner datasets (we use the fancy term improved test–retest reliability).
Usability tests pointed at the same problem: people never trust their first test because they don't feel proficient enough, and end up redoing it to make sure their results are accurate.
How might we ease users' learning curve so that their hearing data becomes reliable quicker, and they feel confident about their skills to perform the test?
To tackle this hybrid challenge of usability & perceived confidence, I spearheaded an iterative prototyping project to find the right form of test instructions.
Early on, usability tests revealed that the winner concept succeeds in engaging the user more than a block of text would, but it also has the potential to induce cognitive overload.
“There's so much to do, listening to the beep, the text keeps changing. I have no idea what the colors might mean, maybe caution, to watch out for something?”
After a few rounds simplification through low and high fidelity prototypes, the design team was ready to present the concept of an interactive tutorial that nudges users to learn the interaction. We were ready to pop the post-release champagne.
Not so fast.
Building real products takes grit and patience to work with unforeseen constraints.
A technical proof-of-concept revealed that the animations were so CPU-heavy on the tech stack that it couldn't play sound any longer.
A meagre MVP and a tech stack rebuild later, our concept got a facelift, and was ready to try again.
This concept gives real-time positive and negative feedback visually.
Succint verbal instructions are unavoidable.
The interaction model is simpler, without different modalities being added to the same UI element.
What makes this concept complex?
The friendly UI is deceptively simple. The challenge lies in refining the underlying logic when positive/negative feedback is triggered. A traditional analysis of success metrics (high conversion, low drop-off rate) can't directly show the benefits of the feature. Yet, some form of small-scale validation has to happen before wide rollout. Just like the recognition of the problem, the validation of the solution needs to be data-driven: a purely qualitative approach doesn't cut it.
To achieve this, we need to dig a level deeper into the generated raw data to measure if the test-retest reliability improves in various hearing ability cohorts.
Giving real-time feedback during the test shouldn't distract users or falsify the test results, so reaction time isn't a good metric: people with different hearing abilities react differently to sounds, therefore faster reaction time may actually mean that the user isn't actually reacting to the sound.
The goal of the hearing test practice feature is to reduce the amount of erratic data recorded by users. By A/B testing the addition of the feature, we can measure whether the rate of erratic behavior decreases significantly. The pre-launch evaluation phase is still in progress.