Large language models are described as deep learning models trained on massive neural networks that can process vast sequences of text and extract meaning fast, and since mid-2024 context windows have expanded to 200K+ tokens and costs have dropped by 80-90%. When applied to healthcare, these models support a wide range of use cases, including answering questions, summarizing text, paraphrasing complex jargon, translating language, using tools, calling external systems, and orchestrating complex multi-step workflows. Medical providers are using large language models to streamline administrative tasks for clinicians who spend roughly 33% of their workday on activities outside of patient care, to manage clinical documentation with retrieval-augmented generation over electronic health records, to detect potential adverse events, and to orchestrate care workflows that identify high-risk patients and manage outreach and escalation.
The article argues that the defining challenge for healthcare deployments is context management, with the quality of outputs described as almost entirely determined by the context provided. Brendan Smith-Elion characterizes this as the context problem and emphasizes that the hardest part is architecting systems that dynamically assemble relevant patient data, clinical guidelines, organizational policies, and real-time information. The rise of Anthropic’s Model Context Protocol in 2024 has standardized how models connect to data sources, but it has also exposed the complexity of integrating electronic health records, claims systems, health information exchanges, and external data while managing permissions, context freshness, and multiple sources for complex queries. Agentic architectures that operate over minutes or hours require persistent, accurate context, and regulators like the U.S. Food and Drug Administration and the Office of the National Coordinator for Health Information Technology now expect clear documentation of data provenance, which pushes organizations to track the complete context fed into Artificial Intelligence systems.
Implementation hurdles span context assembly, model lifecycle, trust, and infrastructure. Organizations must build real-time context pipelines across disparate systems, implement semantic search with vector databases, strategically manage context windows even when they reach 200K tokens, and ensure that lab results, medication changes, and care plan updates remain fresh. They are encouraged to use model versioning, hybrid designs combining general-purpose models with domain-specific ones, and retrieval-augmented generation as a hedge against outdated training data, while also providing explainable context chains, human-in-the-loop review, and detailed audit trails. The piece recommends a Plan, Do, Study, Act framework that starts with designing a context architecture, mapping use cases to data sources and latency requirements, choosing between retrieval-augmented, agentic, or hybrid patterns, then implementing context-first infrastructure, experimenting with multiple models such as Claude, ChatGPT, and Gemini, and using reinforcement learning from human feedback that explicitly evaluates context sufficiency.
Evaluation is framed around context completeness as much as output quality, using expert review of both outputs and their underlying context, and tracking metrics like retrieval latency, relevance, completeness, and freshness while stress-testing edge cases and monitoring for schema or data quality drift. Operationalization then depends on context governance, including monitoring dashboards for context assembly success, alerts on stale or missing data, feedback loops when clinicians override recommendations or workflows stall, clinical advisory oversight, data stewards for context quality, and prepared audit trails for regulatory review. The article concludes that healthcare organizations succeeding with large language models share a common investment in unified health data platforms, vector databases for semantic retrieval, a Model Context Protocol server layer, workflow orchestration, and observability and governance. This infrastructure is described as expensive and complex but necessary for production applications such as prior authorization automation with success rates that now exceed 85% for routine cases, population-scale care gap closure, point-of-care decision support delivered in seconds, and patient engagement agents that maintain context across interactions over weeks, and the author argues that future value will depend less on model choice and more on disciplined investment in context-aware Artificial Intelligence architecture.
