Abuela Assistant: Agentic Healthcare Companion

A bilingual, local-first LangGraph state machine acting as a medical scribe and proactive conversational AI.

Python
LangGraph
OpenAI Whisper
Ollama
DuckDB
Pydantic
Mar 12, 2026

Architecture

Before

Linear Scripts

After

Agentic State Machine

LangGraph

Data Privacy

Before

Cloud APIs

After

100% Local

Secure

Language Support

Before

English-Only

After

Multilingual Support

Seamless Translation

The "Why" Behind the Project

I built this for my own grandma. Doctor visits are overwhelming, but when you add a language barrier to the mix, critical details about medications and follow-ups just get lost. I wanted to build a private, bilingual AI assistant that could passively listen in on her English appointments, extract the structured medical data, and act as a conversational, Spanish-speaking diary when she's at home.


The dual-mode architecture: A passive 'Scribe Mode' for the clinic, and an active LangGraph state machine for the 'Home Mode'.

Agentic Orchestration & LangGraph

To make this work, I had to move past simple linear scripts and build a true Agentic Workflow. The core intelligence is driven by a LangGraph state machine. Rather than relying on rigid `if/else` logic, I built a semantic 'Gatekeeper' using a local Qwen model. When my grandma speaks, the Gatekeeper classifies her intent—even if it's a multi-intent query like, 'My knee hurts, what medicine should I take?'—and dynamically routes the execution flow to the right tools.


LangGraph State Machine

Utilized TypedDict and MemorySaver to maintain conversational context and short-term memory across multi-turn interactions.

Defensive LLM Wrangling

Used Pydantic to force open-source models into strict JSON contracts, guaranteeing predictable, type-safe payloads for DuckDB.

Concurrent Tool Execution

Engineered a routing system that dispatches multiple 'Send' objects in parallel, allowing the AI to query databases and log symptoms simultaneously.

100% Local Privacy

Integrated OpenAI Whisper, Ollama, and DuckDB to run entirely locally, ensuring no sensitive medical audio or text ever hits a cloud provider.

Tags
Data & AI
Platform & Backend
Abuela Assistant: Agentic Healthcare Companion | Luis Tupac