Overview
ModernDoc is an early stage health tech startup that uses AI to help people better understand their symptoms and have more informed conversations with their doctors. Their goal is to reduce misdiagnoses through an intelligent chatbot that makes reliable medical guidance more accessible.
Problem
While ModernDoc delivers quick, personalized health advice, users hesitate to fully trust the AI-generated information without visibility into its sources or a way to verify its accuracy.
Solution
Introduce a source citation feature that gives users immediate insight into where the information is coming from, enhancing credibility and strengthening ModernDoc’s competitive edge.
Impact
Overall improved satisfaction and confidence in the app
4.3
Users gave ModernDoc an average Trust Score of 4.33 out of 5
(as compared to 3.5 for competitors)
57%
57% of users trust ModernDoc more than competitors
+40%
40% increase in the percent of users’s who trust ModernDoc more than competitors
DISCOVER
Visibility and trust in credible sources is critical in driving a competitive advantage for ModernDoc
To guide the design process, we started by interviewing six potential users who have dealt with misdiagnosed or complex health conditions.
The goal was to learn how to attract and retain new users by exploring how potential users currently research health concerns:
What resources do people currently use to research health concerns?
How do they choose those sources?
How do they determine if the information is trustworthy?
How does ModernDoc compare?
Key Insights:
Trust is critical in driving a competitive advantage
Because users rely on ModernDoc for personal, high-stakes health decisions, without trust, they'll turn to other sources.
1
Users Verify
People seek reassurance through cross-referencing from multiple sources.
2
Reputation Counts
People are more likely to trust answers backed by familiar organizations.
3
Transparency is Key
Trust grows when users see “credible” sources, especially with AI solutions.
So what makes sources “credible”?
Surveyed users look for brand familiarity and recently published sources when gauging credibility
To find out what makes sources credible to potential users, I conducted a survey with 23 participants that explored how users evaluate trust in online medical content.
Respondents shared they're more likely to trust information when they can:
View clearly listed citations within the article
Cross-reference information across multiple sources or sites
Recognize the reputation of the website or institution
Confirm the article was recently published
Survey Trust Scores reveal ModernDoc lacks an advantage over its competitors
8.7%
Only 8.7% of respondents felt ModernDoc was at least somewhat more trustworthy than their usual sources.
"Compared to its competitors, I trust ModernDoc…"
MuchMore
0%
SomewhatMore
8.7%
About theSame
56.5%
SomewhatLess
13.0%
MuchLess
4.3%
Without added credibility, users are unlikely to break their existing habits and switch to a new app.
DEFINE
Opportunity through creating transparency
All these findings pointed to a clear desire: users wanted to see source data so that they could validate their trust in modern doc.
USER PROBLEM
Lack of Transparency
Users hesitate to trust AI-generated health advice without clear, credible sources.
BUSINESS PROBLEM
Lack of a Competitive Edge
Without a stronger differentiator, potential users aren't motivated to change their current behaviors and make ModernDoc their go-to source.
How might we create source transparency in order to increase users' trust in ModernDoc's AI-generated health insights and create a competitive advantage?
Goal: Increase perceived credibility (Trust Score) to differentiate ModernDoc from generic search and AI tools by making the sources of information more transparent, recognizable, and easy to verify.
POV: I'm a user struggling to find accurate, personalized health information I can trust
To humanize these insights and guide the design process, I developed a user persona.
Meet Alexis! Alexis struggles with strange symptoms and is overwhelmed by the concerning and often contradictory health information she finds online.
How might we… help Alexis easily find and trust the answers she needs without needing to consult multiple sources?
IDEATE
We knew we wanted to display source material through citations. Our challenge was to surface citations in a way that builds trust without disrupting the conversational experience users expect from an AI chatbot.
AI Competitors are establishing design patterns
AI competitors, such at ChatGPT (Plus) and Perplexity are teaching users to expect features, such as
Inline citations
Simple source buttons
Expandable bottom sheets
Article titles, institutions and/or URLs
These features also aligned with what users prioritized in our earlier survey.
However, these patters aren't mainstream… yet
Only 56% of survey respondents use AI tools for health-related searches and even fewer pay for the versions that showcase these more advanced citation features. Still, these companies have likely conducted significant UX research, giving us an opportunity to build on their insights and adapt them to ModernDoc's needs.
The MoSCoW Method helped define the key features
Based on my user research and competitive analysis, I prioritized key features for an MVP using the MoSCoW method of prioritization.
Low-fidelity sketches let us explore different layouts quickly
Now it was time to start visualizing! I began by sketching low-fidelity ideas for how sources could appear at the bottom of the ModernDoc's messages.
After speaking with the founder, I learned that including logos would be technically complex and potentially costly, so I intentionally avoided them in this round of exploration.
I explored two layout directions:
Condensed
Clean, minimal options using simple text links or buttons
Expanded
More detailed designs that display key information
PROTOTYPE & TEST
Comparative Testing explored how design variations impacted user trust and confidence
To narrow in on the best solution, we selected our top three citation designs and set up a moderated user test to gather feedback.
We showed six participants four different versions of ModernDoc: one baseline version with no visible sources and three versions with different citation display styles.
For each version, users were asked to:
Rate their trust in the information using a Trust Score
Choose the version they preferred most
Status Quo
Static List
Simple Button
Horizontal Scroll
Result: The critical factor was the ability to see and access sources
Despite users expressing UI preferences, these preferences didn't meaningfully impact their trust ratings.
The critical factor that increased trust was the ability to see sources and access those sources via links.
Result: Users liked and trusted both the Simple Button and Horizontal Scroll
Of the options shown, users most often said their favorite was the Simple Button. Three participants ranked it as their top choice, and two others included it among their top selections.
However, while the Simple Button was most preferred, the Horizontal Scroll was the second most preferred and received a slightly higher average Trust Score due to one person who preferred the Horizontal Scroll most and rated it slightly higher.
Average Trust Scores
3.98
StatusQuo
3.76
CompetitorOptions
3.98
Static List
4.60
Simple Button
4.63
Horizontal Scroll
NOTE:
Trust Scores were the same for the Simple Button and Horizontal Scroll options for all testers except one who preferred the Horizontal Scroll most and gave it a slightly higher trust score.
Would an updated design surface a clear winner?
The Simple Button was the favorite, but we wondered… would users prefer the Static List if it included clickable links? Instead of reintroducing a long list, I combined the clarity of the list with the simplicity of the Horizontal Scroll.
Before
Removed image to simplify layout and mirror the list-style version with link emphasis
Repositioned title to the top to showcase title over source
Added “See all sources” button to keep scroll manageable when many citations are present
After
Surprise! ModernDoc sources are primarily journal articles, not familiar websites
In the first round of testing, I used Perplexity-style sources as placeholders. However, after speaking with the founder, I learned ModernDoc surfaces content from Pinecone, its vector database, which pulls credible but less recognizable sources such as medical journals.
Issue:
Four of six users said source familiarity influenced trust. So while journal articles are highly credible, they're not as recognizable as names like John Hopkins Medical or the PMC.
Iterations accounted for this new information
To adjust for this constraint, I changed how the sources were shown in the bottom sheet to emphasize titles over institutions and provide the institution when it is available.
I also added the Publication Date as four out of six user testers expressed the desire to see this for added credibility in the Simple Button design.
Before
Focus on title over institution
Provides institution name if available
Adds publication date
After
A second round of comparative testing evaluated the two top updated designs
Simple Button
Horizontal Scroll
Results: Adding citations continued to display an increase in trust relative to competitors
When asked directly how much users trusted ModernDoc compared to competitors, 57% (4 out of 7) said they trusted ModernDoc more, 43% (3) said about the same, and no one reported trusting it less despite some concerns about trusting AI-generated responses overall.
4.3
Average Trust Score(Score out of 5)
57%
Trust ModernDoc more than competitors
+40%
Increase in percent of users who trust ModernDoc more than competitors
Results: Switching citations to focus on title and URL did not greatly impact trust
Once again, ModernDoc (with citations) received the highest average trust score compared to competitors participants used.
These trust scores were also similar to the previous test, indicating that switching the focus to title and URL did not seem to greatly impact trust.
Results: Users trusted both designs equally but still showed a preference for the Simple Button design
Both designs performed well for different reasons. However, there was a slight preference for the Simple Button.
Feature | Simple Button | Horizontal Scroll |
|---|---|---|
User Focus | Encourage focus on answers | Emphasized research/citations |
Ease of Use | Clean, minimal, less distracting | Easy to scan, visible citations |
Trust Impact | Trust not impacted by hidden sources | Trust not significantly increased |
Perception | Streamlined & test-friendly | Sometimes read as ads |
Engineering Effort | Easier to implement | Harder to implement |
FINAL DESIGN
Winner: The Simple Button!
After reviewing the results with the founder, we decided to move forward with the Simple Button.
While the data didn't reveal a definitive winner and showed that both designs performed similarly, each with its own tradeoffs, the Simple Button revealed a slight edge in multiple tests and came with the added benefit of being simpler to implement from an engineering standpoint.
REFLECTION
What’s next?
Boosting trust was just the first step. We saw encouraging signs in testing, but the real impact will only become clear through user behavior.
Once citations are implemented, we will monitor changes in monthly active users (MAU). If the new design fosters real trust and drives users to rely on ModernDoc for their health research, we expect to see an increase in MAU.
Looking ahead, we'd like to explore more ways to increase trust and highlight the credibility of less familiar sources. Future iterations could include features like a confidence score or visual indicators showing whether a source is peer-reviewed, award-winning, or pre-published.
Lessons Learned
The Founder's Thoughts
fill this in!!!
Philip Cortes
Founder & CEO
