Mitigating Uncertainty in voice only searches for older adults

Designed a Voice-based based cue that positively influences and supports older adults in evaluating quality of responses to health-related advice queried by voice assistants.

Problem

An increasing number of older adults use voice assistants to answer health-related queries but are unsure whether the responses are credible due to the limited information presented by the voice assistant. The lack of rich cues in voice assistants to evaluate credibility makes it more likely for people, especially older adults, to make decisions based on misinformation, which can be very dangerous when applied to health-related contexts.

Design Outcome

To address this problem, I designed a voice-based default nudge that alerts the user about the sensitive nature of their query and cues the user to choose one of two options that can help clarify the credibility of the information. By setting the nudge as the default for health-related queries, we aim to increase its impact to positively influence older adults to be objective in their evaluation.

• Nudge that brings to top of mind the sensitive nature of the user’s query
• Presents options for quick assessment as the defaults to nudge users to seek more information

Curates most important data points to help user decide - Can I trust this information?

• What kind of organization is the publisher?
• Who is the author and are they qualified?
• Is the content outdated?

Provides an summary of comparison of the responses found other sources

• Highlights what is similar or different

Project Context

The project lead by Dr. Robin Brewer aims to investigate voice technologies for helping older adults navigate uncertain information in decision making. The project was divided into 4 phases and I was recruited when the project was in phase 2.

My role was to collaborate in designing solutions based on the findings and recommendations in the pilot research from phase 1.

How did we arrive at the design?

Understanding the Pilot Research

We started by analyzing and understanding the findings and recommendations from the pilot research with conducted by Dr. Brewer and her team.

Findings

Pain points

• Unsure how to behave when they encounter unhelpful voice assistant responses in sensitive contexts
• Participants either disregarded or over trusted the response or misinterpreted it by filling in the gaps from their own knowledge.

Older adult’s expectations and behavior

• Expect to start and complete their search process with voice assistant. Preferred responses that encouraged a follow up queries or conversations
• Gaps in digital literacy posed a barrier
• More susceptible to misinformation because they are biased by preexisting beliefs and interpersonal trust when evaluating online information

Recommendations

• Provide options for quick quality assessments to encourage healthy search behavior
• Allow users to ask questions about sources though conversation
• Use voice-based nudging approaches to encourage reflection

Framing the Problem

Debating the choice architecture

We asked ourselves the questions -

• What kind of nudge intervention should we design?
• Are we changing the design of the system or the behavior of the users?

We eliminated approaches where

• User has no transparency on how credibility was determined by the assistant
• The AI made the user’s decisions that it determined was best for the user
• No opt out mechanism

Instead selected on approaches which

• Provided a way to make quick assessments easily
• Role of the assistant - a guide rather than a decision maker

Defining the Interaction

We brainstormed on

• Where should the nudge intervention take place in the interaction?
• What behavior do we want the user to display after being nudged?
• What medium - Verbal or Non-verbal - should we design the nudge?

High Level Conversation Flow

We decided to explore both mediums for the nudge.

👥

I took responsibility to design the verbal nudge while Meredith, who has experience with music explored the non-verbal nudge design.

Exploring Nudge Strategies

I referenced the papers on nudge theory by Richard Thaler and Cass Sunstein to explore different nudge strategies and prompts that would be effective.

Exploring Assessment Options

Brainstormed options older adults would be looking for to make quick assessments based on research papers about older adult’s online search behavior. I wrote sample dialogues to explore how the assistant should provide the options in the conversation.

Secondary Research

Based on feedback after showcasing my work in progress concepts to the team, I realized I needed to make evidence-based choices to narrow down choices. Since formal primary research was out of scope, I looked into literature.

Open questions

• What information would help the user verify credibility?
• What kind of language is used to talk about credibility?
• How can we narrow the prompts to use in the nudge?

Literature Review

I looked at literature from journalism and misinformation research to answer our questions. Additionally we conducted an informal survey to understand what the terms used in literature to define credibility mean to people.

Key takeaways

💡

People associated the word “Believable” with high confidence levels and truthfulness. So “Believable” would likely elevate the state of doubt and uncertainty the when nudged leading to information seeking behavior.

💡

Supporting users to learn more about the publisher and author is recommended to objectively evaluate credibility.

Narrowing Down & Refining

Based on what I learned, I narrowed down to

1. Two nudge strategies to consider
2. Final nudge prompt for each strategy
3. Three evaluation options

Which nudge would be more effective?

We evaluated the two nudge strategies against our goal. Default nudges are more powerful because of the status-quo bias in people including older adults where we perceive default options as the recommended path.

Defining the Content & Final Flows

Based on the research, we decided to provide options to ask the assistant the below clarifying questions.

1. Who is the publisher what type of organization do they run?
2. Who wrote the article and what are their qualifications? When was it published?
3. See what other sites are saying. Do they provide the same answer or something different?

I started by experimenting with different ways to design, structure and templatize the content.

Prototyping & Final Designs

I used voice flow to prototype the flow of the interactions with the voice assistant. We had written out two scenarios based on which I wrote the content using the defined structure.

Next steps

We are currently in the process of planning user testing studies with older adults to evaluate both prototypes - verbal and non-verbal nudge.

Reflection

When ChatGPT and other AIs were in the limelight, I was so surprised to see that people were asking - Where did the AI get this information? Can we trust this info? It was interesting to see the behaviors and needs expressed in the state of uncertainty at a large scale. It was both reassuring and scary. Scary because as Generative AI becomes more pervasive in information systems, there will be less transparency and more uncertainty. Reassuring because people want a better way to interact with AI information systems. While the project is for Older adults, the findings can inform designs for a larger population. I am excited to continue supporting Dr. Brewer on this project and looking forward to putting the designs in front of users.

ON THIS PAGE