The Hallucination Problem Nobody Wants to Talk About
Most healthcare organizations evaluating AI for medical coding focus on automation rates and denial reduction. Far fewer ask a simpler question: how often does the AI invent codes that don’t exist—or codes that exist but don’t reflect the documented diagnosis?
That question got a concrete answer on July 7, 2026, when emtelligent released head-to-head benchmark results comparing its Medical Language Engine against leading general-purpose large language models (LLMs) on real hospital discharge summaries. The findings are uncomfortable for anyone betting on off-the-shelf AI for clinical coding work.
In testing across 272 real hospital discharge summaries and a dataset of nearly 79,000 labeled medical concepts, the best-performing general LLM achieved only a 55% F1 score—and fabricated nearly one in three medical codes. Retrieval-augmented generation (RAG), the technique most commonly proposed as a fix, reduced hallucinations but crashed accuracy further, to a 22.64% F1 score. Fewer than one in four relevant clinical concepts were identified.
emtelligent’s Medical Language Engine, by contrast, scored 89.85% F1 with zero hallucinations and identified all four relevant clinical concepts in the evaluation set.
Why General-Purpose LLMs Fail at Medical Ontologies
The root problem is structural. General-purpose LLMs are trained to generate plausible text, not to validate outputs against a finite, authoritative set of clinical codes. When asked to map a clinical narrative to SNOMED CT, ICD-10-CM, RxNorm, or LOINC, they don’t check their answers against those ontologies—they generate text that looks like valid code output.
“Understanding medical ontologies and terminology is beyond the ability of general-purpose LLMs,” said Tim O’Connell, M.D., CEO and co-founder of emtelligent. “And since LLMs don’t answer ‘I don’t know’ to queries, they can fabricate answers without checking them against accepted medical ontologies. Those coding hallucinations put patients, revenue, and compliance at risk.”
The RAG results are particularly telling. RAG is supposed to ground LLM outputs in retrieved reference material—in theory, that should help coding accuracy. But the emtelligent study found that RAG-assisted LLMs reduced hallucinations at the cost of recall: the model became more conservative but missed far more relevant clinical concepts, dropping to a 22.64% F1. That’s not a usable accuracy level for production coding work.
What This Means for CDI Teams
Clinical documentation integrity (CDI) depends on accurate capture of diagnoses, procedures, complications, and comorbidities at the level of specificity that drives correct MS-DRG assignment, accurate HCC coding, and defensible audit trails. A hallucinated code—one that doesn’t reflect the actual clinical record—creates compliance exposure at every downstream touchpoint.
The emtelligent benchmark used SNOMED CT, one of the most granular clinical terminologies in use. SNOMED CT is increasingly relevant to CDI workflows because it underpins concept mapping between clinical narratives and coding systems. A system that cannot reliably traverse SNOMED CT is a system that cannot reliably support CDI.
The hallucination risk isn’t just about claim accuracy. Consider the downstream effects:
- Fabricated ICD-10 codes that don’t match documented diagnoses create OIG audit exposure under the medical necessity standards that auditors use.
- Missed clinical concepts mean missed CC/MCC captures, lower MS-DRG weights, and lost reimbursement that documentation actually supports.
- For HCC coding in Medicare Advantage, a hallucinated HCC submission is a false claim under RADV audit scrutiny.
- For quality reporting (e.g., PSI, HAC), an incorrect diagnosis code can shift a patient event into or out of a reportable category, affecting hospital ratings.
The Architecture Behind the Accuracy Gap
emtelligent’s Medical Language Engine is built differently from general LLMs applied to clinical text. It identifies clinical entities, codes them to accepted ontologies (ICD-10, SNOMED CT, RxNorm, LOINC), and—critically—captures the assertions and relations that give those entities clinical meaning. A diagnosis coded as “present on admission” versus “resolved” versus “suspected” carries different coding consequences. A general LLM typically doesn’t make that distinction reliably.
Three Integrated Layers
The platform’s architecture combines three components that address different failure modes. Document AI handles the intake problem—splitting, classifying, and performing OCR on bundled PDFs, faxes, and scanned records before any NLP begins. The Medical Language Engine performs entity extraction, ontology coding, and assertion/relation capture. AI Chart Review provides a human-in-the-loop interface for coders, CDI specialists, and compliance teams to review, validate, and trace every output back to the source documentation.
The platform also now includes Model Context Protocol (MCP) server functionality—meaning healthcare organizations can integrate their own LLMs and supply structured, accurate clinical data from emtelligent as context. Instead of asking the LLM to do the coding work, you give it pre-validated clinical data to reason on. That’s a meaningful architecture shift from RAG.
A Benchmark Number That Should Change Procurement Conversations
A 55% F1 score with 1-in-3 hallucinations is not a vendor limitation that gets patched in the next model release. It reflects a fundamental mismatch between what general-purpose LLMs are optimized for and what clinical coding actually requires. The coding layer in healthcare has to be right—not approximately right, not right on average. A hallucinated code on a single high-cost claim can trigger a recovery audit across an entire encounter category.
The emtelligent study used 272 real discharge summaries, not synthetic test cases. That matters because clinical documents are messy—abbreviations, implicit findings, complex comorbidity language, and documentation across multiple notes that need to be reconciled. The benchmark reflects production conditions, not laboratory ones.
For coding and CDI leaders evaluating AI tools, the study surfaces a question worth pressing vendors on: what is your hallucination rate on real hospital discharge summaries, tested against a validated clinical ontology? If a vendor can’t answer that question with a published methodology, that’s a signal worth acting on before go-live.
What Coders Should Take Away
The emtelligent results don’t argue that AI has no place in medical coding—they argue that the wrong kind of AI creates risk that’s hard to detect until an audit surfaces it. Coding hallucinations don’t trigger edit checks the way claim formatting errors do. A code that’s clinically incorrect but structurally valid will pass claim scrubbing. The only things that catch it are human review, payer algorithms, or OIG analysis.
Clinical-grade AI—purpose-built for ontology mapping, trained on annotated medical data, and designed to output structured findings that can be traced back to source text—closes that gap. General-purpose LLMs, including RAG-enhanced configurations, don’t appear to be there yet.
As more RCM teams move AI coding tools from pilot to production, governance frameworks need to include explicit hallucination testing before deployment. The emtelligent benchmark provides one model for how to run that evaluation. Teams that skip it are accepting a compliance risk they may not fully understand.
For organizations exploring what accurate, auditable AI looks like in practice, Medikode’s automated medical coding platform is built on the same principles: every code is traceable, every output is explainable, and accuracy is measurable against real clinical documentation.