The opportunity…

Our program for people managing chronic conditions includes health safety alerts. These alerts use an algorithm that looks at blood pressure, pulse, and weight readings members log as well as trends in readings over time to identify when a member may be immediately unsafe.

Version one of the alerts system surfaced abnormal and critical alerts to the member’s primary care provider and a member of our company’s care team. This setup required a member of our company’s care team to call the member to gather more information and determine next steps to make sure the member was safe. Being able to accurately triage alerts with minimal human involvement could lower costs for our company and for our clients. It was essential that the solution still be effective at identifying potential acute episodes and reducing hospitalizations, and a digital-led solution would need to be safe and usable for our members.

To design a solution, the team needed to understand…

This feature set was on the product roadmap for post-MVP release, and we had done a lot of the initial UXR and design legwork in the foundational phase. I took time with the PM to work together on defining the key user needs and risks related to this feature set. Then, we shared them our regulatory and quality stakeholders to lay a solid foundation for this research to tee us up for summative testing the final solution. We coordinated with engineering and design to ensure that the UX we were testing was feasible with the development resources available. Based on this, we aimed to address the following research questions:

So, I developed a research plan…

with an iterative design. We had limited time to implement the research and finalize designs before the design handoff. An iterative approach would unblock design and content stakeholders and enable them to implement changes based on our learnings as we went, instead of having to wait for me to analyze what we learned and put together the UX research deliverables. A key risk to user safety in using a digital-led solution to triaging alerts was misunderstanding the content and providing information that could lead to inaccurately deescalating an alert. I included content testing methods to assess the understandability and comprehension of key content and questions.

Because we were implementing this research during the COVID-19 pandemic, I implemented the research remotely using the User Interviews platform for recruitment and the dscout platform for sessions.

We implemented…

I got buy-in from my key stakeholders who would need to work together to implement an iterative testing approach. The designer and content strategist agreed to participate in debriefs after every 1-2 sessions to talk about what we had learned and decide whether to iterate, and the broader team of stakeholders such as our PM, engineer, and clinical lead agreed to participate in debriefs as needed to provide their input.

This wasn’t cut and dry RITE-style usability testing, so a hard and fast rule of iterating after a specific number of sessions or iterating the first time an issue came up wasn’t appropriate for some of the more nebulous items we were evaluating like content comprehension. I created this decision tree to help the team navigate deciding when we should iterate.

We identified key research themes…

• A digital-led health alert experience aligned well to users’ mental models around managing health-related risk.

• Not all patients are the same. What’s considered ‘okay’, triggers and symptoms, and what an acute episode looks and feels like can vary widely from one patient to another.

• Participants didn’t want the experience to be overly alarmist, but they also didn’t want us to keep any information that might help them better assess or understand their health from them.

• Not everyone understands their condition, and users want to build their understanding of their own bodies, symptoms, and triggers to help them better manage their condition.

• For people managing chronic conditions, very few situations are black and white. Each person has their own experiences, context, and beliefs about what is an dis not okay for them. Participants wanted help interpreting data and identifying things that might warrant further attention, as well as guidance and suggestions on what they should do next, but would also use their own judgement.

To socialize learnings…

And as a result of the research…

Based on our learnings, design and content made considerable changes to the UX throughout the iterative testing process. By the final few sessions, we were seeing comprehension and task completion at satisfactory levels to give positive signals for user safety and anticipating passing summative testing against regulatory standards.

There were some clear improvements that could be made to the experience that were out of scope to address in the GA release based on development resources, which informed feature prioritization for the product roadmap - for example, offering an SMS-based version of the symptom collector flow, and increased personalization like member-specific targets and thresholds for ‘normal’ and abnormal readings surfaced in the digital experience.

Learnings also continued to inform a clear and distinct design, content, and UX strategy. For example, based on these learnings and building on foundational learnings, the roadmap prioritized: supporting members with real-time data interpretation; calling out things that might be off in the UX while developing a design approach to balance that with ensuring users aren’t triggered into a state of panic or alarm; and education to build trust with users. For example, ensuring the UX clarifies things like how we decide the range/threshold for a normal vs. abnormal reading, what different symptoms have to do with heart failure, and what else could trigger a high BP reading (i.e. a salty meal, stress, missing a medication, etc.).