AI xPress

AI and computer science research.

Back to Foragentis

Healthcare & Bio AI

Most electronic health record foundation models are trained on narrow data and tested on tasks that say little about hospital usefulness, review finds

Healthcare & Bio AI November 1, 2026

A bead model of a protein on a desk

A 2023 review in npj Digital Medicine found that most foundation models for electronic medical records were trained on small, narrowly scoped datasets or on broad public biomedical text, and were tested on tasks that said little about their usefulness to health systems. Researchers at Stanford University, Stanford Medicine, and Stanford Health Care examined 84 models trained on non-imaging electronic medical record data. Foundation models are large AI systems, such as ChatGPT and AlphaFold, that learn from broad data and then adapt to many tasks.

What did the review examine?

The paper is a narrative review of 84 foundation models trained on non-imaging electronic medical record data. That scope covers clinical text or structured data, not medical images. The authors sorted the models into a taxonomy, or classification, based on their architectures, training data, and possible use cases.

What did the researchers find?

Most models were trained on small, narrowly scoped clinical datasets such as MIMIC-III, or on broad public biomedical corpora such as PubMed. The authors found that the evaluation tasks used for these models did not give meaningful insight into how useful they would be in a real health system.

Why does that matter?

Hospitals and clinics need to know whether a model would improve patient care or operations. The review says the current tests do not answer that question well. The authors propose an evaluation framework that is more closely tied to metrics that matter in healthcare.

What do readers commonly ask?

What is a foundation model? A large AI system trained on broad data and adapted to many tasks. The paper considers foundation models developed for electronic medical records.

What data were the models trained on? Most were trained on small, narrowly scoped clinical datasets such as MIMIC-III, or on broad public biomedical corpora such as PubMed.

What did the authors propose? They proposed an evaluation framework for measuring the benefits of clinical foundation models that is more closely connected to healthcare metrics.

Limits and what's next

The review covers 84 foundation models trained on non-imaging electronic medical record data, meaning clinical text or structured data. It does not include imaging models. The abstract does not say how the 84 models were selected or how the authors counted training datasets and evaluation tasks. This is one narrative review, and the abstract reports no independent replication of its findings. The authors propose an improved evaluation framework, but the abstract does not report results from applying it.

Source: Michael Wornow et al., “The shaky foundations of large language models and foundation models for electronic health records,” npj Digital Medicine, 2023.