Faces of Digital Health

AI models are "eager to please" — and in healthcare, that's a liability. So what should LLMs never be allowed to do in clinical software?

Three years after GPT-3 reached the public, frontier models have largely converged in capability. In this episode, Robert Tovornik, Innovation Lead at Better — the healthcare IT company building on openEHR — explains why the real differentiator in healthcare AI is no longer the model but the data layer underneath it. He makes the case for keeping clinical coding and terminology in deterministic systems, confining LLMs to retrieval and orchestration, and validating AI the way you'd come to trust a colleague: through experience, not certification.

Guest: Robert Tovornik, Innovation Lead, Better

What the conversation covers:

- Why frontier LLMs are converging — and why context now matters more than model capability

- What AI should never do in clinical software: inference vs retrieval

- Why ICD-10 and SNOMED coding should stay in deterministic systems, not ChatGPT

- How to validate non-deterministic AI systems when unit tests no longer work

- Automation bias: what happens when users stop checking AI outputs

- Conversational EHRs — solving the "missing button" problem in clinical interfaces

- Vibe coding vs regulated clinical software: why one iterates in hours and the other in years

- An ambient AI scribe built in two weeks — deployed in India, stalled in Europe

- EU AI Act, data residency laws, and the cost of compliance

- Digital twins, ambient AI, and what hospitals should invest in before deploying AI

Faces of Digital Health explores how healthcare systems around the world adopt digital technologies and AI.

🔗 Website: https://www.facesofdigitalhealth.com

🎧 Spotify: https://open.spotify.com/show/4cElKJHrauyP6QJQaCkvdY

🎧 Apple Podcasts: https://podcasts.apple.com/gb/podcast/faces-of-digital-health/id1194284040

📰 Newsletter: https://fodh.substack.com

💼 LinkedIn: https://www.linkedin.com/company/faces-of-digital-health

#healthcareAI #openEHR #digitalhealth #healthIT #EHR #clinicalAI #healthcareinnovation

02:20 Three years of GPT: from model capability back to data and context

04:26 How AI changed software development inside an openEHR vendor

06:13 Clients now arrive with AI-informed (and misinformed) requirements

09:05 Why clinical coding belongs in deterministic systems, not ChatGPT

11:27 "Eager to please": why LLMs shouldn't be trusted with inference

14:03 Validating non-deterministic AI when unit tests no longer work

16:50 How do you trust AI? The same way you trust a colleague

18:03 Regulation, compliance costs, and the automation bias problem

20:09 Conversational EHRs and the missing-button problem

23:02 Vibe coding vs iterating regulated clinical software

25:23 Users are building AI experience faster than health systems

27:50 An ambient scribe built in two weeks — adopted in India, stalled in Europe

29:16 Data quality as the differentiator between good and bad AI systems

31:10 Ambient AI, operation prep, and the digital twin horizon

AI couldn't cure his mother's stage 4 cancer. It caught three near-fatal errors, found a same-day appointment, and helped her leave on her own terms.

When Pratik Desai's mother was diagnosed with stage four duodenal adenocarcinoma — a rare cancer with roughly 3,000 US cases a year — she was nearly discharged without an oncology appointment. Over the next 76 days, Desai used AI at her bedside, from 5am to 10pm, to understand each report, prepare for every appointment, and push a stretched health system to move at the pace her diagnosis demanded. This is a frank account of where AI helped, where it didn't, and the line he refuses to cross.

This is a 1:1 interview in The Agentic Patient — a Faces of Digital Health series on how patients and caregivers actually use AI: which tools, which prompts, and which guardrails.

GUEST

Pratik Desai — New Jersey-based AI practitioner; caregiver and builder of a free, local AI tool for patients

HOST

Tjaša Zajc — Founder & host, Faces of Digital Health / The Agentic Patient

WHAT THE CONVERSATION COVERS

- Using AI to interpret a biopsy report and push for a same-day "stat" CT scan

- Why AI and the doctors agreed on the care — and clashed on the speed

- Finding a same-day oncology appointment through an AI-assisted network search

- An error-riddled CT report the AI refused to read — and what it did to trust

- Running three Claude "personas" as built-in second and third opinions

- A local, open-source AI tool that keeps medical data off the cloud

- How to prompt as a patient or caregiver: awareness, knowledge, advocacy — not diagnosis

- Where AI failed him: prognosis, and the rule he broke under pressure

- Defining quality of life when the outcome is already known

CHAPTERS

0:00 How patients use AI — and the guardrails

1:20 Day one: a healthy mother, a diagnosis no one would name

3:34 The first prompt, and pushing for a stat CT scan

7:43 Using AI in the open: agreement on care, friction on speed

9:35 The counterfactual: 76 days with AI at the bedside

12:40 Finding a same-day appointment through a network search

13:40 The CT report the AI refused to read

15:50 When trust erodes: good faith, not competence

18:41 Why switching hospitals wasn't an option

21:54 Defining quality of life: her three goals

28:27 Three Claude personas, and a local private tool

35:12 How to prompt: awareness, knowledge, advocacy — not diagnosis

37:54 Where AI fell short, and the closing asks

THE AGENTIC PATIENT SERIES

New to the series? Start here → [PASTE PREVIOUS AGENTIC PATIENT EPISODE LINK]

All episodes → https://www.facesofdigitalhealth.com/agentic-patient-blog

MORE FROM FACES OF DIGITAL HEALTH

🌐 Website: https://www.facesofdigitalhealth.com

📨 Newsletter: https://fodh.substack.com

🎙 Podcast (Apple): https://podcasts.apple.com/gb/podcast/faces-of-digital-health/id1194284040

💼 LinkedIn: https://www.linkedin.com/company/faces-of-digital-health

Pratik's tool Regana: https://github.com/RaganaCorp/openhealth-prototype-1

#DigitalHealth #HealthAI #AgenticPatient #PatientAdvocacy #AIinHealthcare #CancerCare #Caregiving #FacesOfDigitalHealth

AI ethicist Jess Morley: these chatbots are giving medical advice — so regulate them as medical devices.

Part of The Agentic Patient, a Faces of Digital Health series on how patients actually use AI — which tools, which prompts, which safeguards. In this episode, host Tjaša Zajc sits down with Dr Jess Morley, Associate Research Scientist at the Yale Digital Ethics Center and a former AI subject-matter expert at the UK Department of Health and Social Care, for a clear-eyed account of where health AI is going wrong — and how to use it well anyway.

Morley argues we systematically overestimate what these tools can do and underestimate the harm. She makes the case for "skeptical optimism," explains why bioethics principles built for one-to-one care break down against many-to-many AI harms, and reframes ambient scribes as inference engines rather than transcription services — with real consequences for coding, billing and patient records. Then she gets practical: the guardrails, prompts and habits patients (and clinicians) can use today.

Guest: Dr Jessica Morley — Associate Research Scientist, Yale Digital Ethics Center; formerly UK Department of Health and Social Care and the Bennett Institute, University of Oxford.

What the conversation covers:
- Why "skeptically optimistic" is the honest position on health AI
- AI adoption as "a hammer looking for nails" — and what needs-led design would look like instead
- OpenEvidence, EU rules and the question of regulatory capture
- The DeepMind–Royal Free case and why law alone isn't enough
- Beneficence, non-maleficence, autonomy, justice — and where they fail for AI
- Ambient AI scribes, miscoding, billing inflation and phantom tests
- Paid vs free models and the widening access gap
- The "ask why" rule and knowing when to walk away from a chatbot
- Red-teaming your own assumptions and playing models off each other
- Building a personal "harness" with skills so AI works from your history
- The last-mile problem and the case for regulating LLMs as medical devices
- Whether AI is narrowing how clinicians think

Chapters:

02:50 — Intro: The Agentic Patient and the case for skeptical optimism
05:52 — "A hammer looking for nails": adoption pressure without a plan
07:25 — OpenEvidence, EU rules and regulatory capture
09:42 — The DeepMind–Royal Free lesson: why law needs ethics
13:29 — The bioethics principles and what they were built to do
19:40 — Autonomy, consent and the ambient-scribe problem
21:49 — Scribes as inference engines: miscoding, fraud and phantom tests
29:06 — Paid vs free models and the access gap
33:25 — Using AI safely: the "ask why" rule
37:38 — Knowing when to walk away: engagement design and degradation
44:58 — Red-teaming and playing models off each other
49:00 — Harnesses and skills: making the model work for you
51:38 — The last-mile problem and regulating AI as a medical device
58:00 — Does AI narrow the clinician's mind?

The Agentic Patient series: https://www.facesofdigitalhealth.com/agentic-patient-blog
Website: https://www.facesofdigitalhealth.com
Newsletter: https://fodh.substack.com
LinkedIn: https://www.linkedin.com/company/faces-of-digital-health

98% of patients welcome AI in their care — and still want a human in charge.
That tension ran through the OECD and Spanish Ministry of Health conference on scaling AI in health (Madrid, late May 2026), and it frames this episode of Faces of Digital Health. Out of 38 OECD countries, only seven have a formal AI strategy and just over a tenth run workforce upskilling programmes — the ambition is outrunning the institutions meant to govern it. Host Tjaša Zajc brings together voices from across the conference to ask what actually has to change: regulation, trust, who gets a seat at the table, and the parts of the agenda nobody is funding.
Featuring:
- Eric Sutherland — Senior Economist, OECD
- Aferdita Bytyqi — Executive Director & Founding Partner, Digital Transformations for Health Lab (DTH-Lab)
- Erza Selmani — Research Fellow, DTH-Lab
- Valentina Strammiello — Executive Director, European Patients Forum (EPF)
- Dr Ricardo Baptista Leite — CEO, HealthAI (the Global Agency for Responsible AI in Health)
- Dr Persephone Doupi — Senior Medical Officer, Finnish Institute for Health and Welfare; President, European Federation for Medical Informatics (EFMI)
What the conversation covers:
- Why trust — not capability — is the binding constraint on health AI adoption
- The OECD readiness gap: AI strategies, HTA frameworks and workforce upskilling
- How patients really feel about AI: consent forms, transparency, and keeping clinicians central
- Why youth health and wellbeing keep getting left out of AI governance frameworks
- Five recommendations to make the EU AI Act work for health and competitiveness
- Coordinating the EU AI Act, MDR/IVDR and the European Health Data Space
- Health technology assessment and reimbursement as the real barriers to scale
- AI literacy and prevention: the most underweighted lever in the room

Chapters:
0:10 — Welcome: AI in Health & the 2026 OECD Conference in Madrid

0:25 — Key Stats: Only 7 of 38 OECD Countries Have a Formal AI Strategy

2:10 — Eric Sutherland (OECD): We're Not Using Data as Effectively as We Could

3:11 — Afrodita & Erza (DTH Lab): Youth Health Is Missing from AI Governance Frameworks

5:12 — Valentina Stramello (EPF): 98% of Patients Are Positive About AI, But Trust Requires Transparency

7:14 — Dr. Ricardo Baptista Leite (Health AI): 5 Recommendations to Fix EU AI Policy for Health

10:53 — Persephone Doupi (EFMI): We Must Prioritize AI Literacy and Shift Healthcare Toward Prevention

—
🎧 Listen: https://www.facesofdigitalhealth.com
📩 Newsletter (incl. written OECD conference summary): https://fodh.substack.com
💼 LinkedIn:https://www.linkedin.com/company/12594967/
🌐 Site: https://www.facesofdigitalhealth.com
#DigitalHealth #HealthAI #AIinHealthcare #HealthPolicy #EUAIAct #EHDS #ResponsibleAI #PatientVoice #HealthTechAssessment #HealthTech

Doctors are using ChatGPT in clinic right now — and some of them don't care about privacy. Three operators on what that means for healthcare AI.

Recorded live at health.tech in Basel, this panel from Faces of Digital Health unpacks the convergence reshaping clinical software: ambient AI scribes, agentic AI in healthcare, on-device LLMs, and the regulatory drag (MDR, EU AI Act, EHDS) that is widening the gap between what clinicians actually use and what hospitals are allowed to buy.

Host Tjaša Zajc is joined by:

Jonathan Bringas — CEO & Founder, Lapsi Health (Kaiku: FDA-cleared AI stethoscope, ambient scribe and clinical assistant in one device)

Blaž Triglav — CEO, Mediately (drug information platform, 1M+ HCPs across Europe)

Amanda Herbrand — Clinical data modelling consultant, formerly University Hospital Basel

What the conversation covers:
— Why EHR data fragmentation is the precondition AI hasn't solved
— Shadow AI: why clinicians trust ChatGPT more than enterprise tools (and the agency hypothesis behind it)
— The convergence of stethoscopes, scribes, drug information and decision support into one workflow layer
— ROI in healthcare AI: financial, time, clinical accuracy — and Herbrand's fourth dimension, user satisfaction
— "Doctors were the original vibe coders": the 2,000 Excel spreadsheets running European hospitals
— Why FDA-cleared beats MDR in 2026 sales cycles, and what Chile's regulatory minimalism tells us
— The asymmetry that will break European medtech: applicants using AI to build, regulators forbidden from using AI to assess
— On-device AI, ambient computing, AGI in clinical workflows — and the de-skilling risk no one wants to discuss

⏱ Chapters

00:00 — Opening: AI agents, vibe coding, and what doctors actually want
01:30 — Data fragmentation: the precondition AI hasn't solved (Amanda Herbrand)
02:30 — Keiku: collapsing stethoscope, scribe and assistant into one device
05:15 — The convergence reshaping healthcare AI — and the shadow AI in clinic
07:30 — Why doctors trust ChatGPT more than enterprise tools: the agency hypothesis
10:30 — ROI: financial, time, clinical accuracy — and Herbrand's fourth dimension
15:30 — Choosing solutions: modular requirements and FDA-cleared moats
19:30 — EHDS and the missing connector in European standardisation
21:00 — "Doctors were the original vibe coders": the 2,000 spreadsheet problem
24:30 — The two-speed world: regulated medicine vs the Wild West
28:00 — Why Chile's regulatory minimalism beats Europe's MDR
30:30 — Agentic AI vs regulators: the asymmetry that will break European medtech
33:30 — On-device AI, AGI, and the deskilling no one wants to discuss

🎧 View the video podcast: https://www.youtube.com/watch?v=fciFwMmIfRc&t=174s
📩 Newsletter: https://fodh.substack.com
🔗 LinkedIn:   / dashboard  
🌐 facesofdigitalhealth.com

#HealthcareAI #DigitalHealth #AmbientAI #AgenticAI #ClinicalAI #EHR #EHDS #MedTech #HealthTech