Healthcare & Bio AI
Med-BERT improved disease prediction accuracy in two disease prediction tasks
In a 2021 paper in npj Digital Medicine, researchers at The University of Texas Health Science Center at Houston and Peng Cheng Laboratory presented Med-BERT. The model learns contextualized embeddings, numerical representations of medical records that depend on surrounding information, from structured electronic health records. After pretraining on 28,490,650 patients, Med-BERT raised the area under the receiver operating characteristic curve, a standard measure where higher scores mean better prediction, by 1.21 to 6.14 percent in two disease prediction tasks from two clinical databases.
On tasks with small fine-tuning training sets, the pretrained Med-BERT raised AUC by more than 20 percent, or reached an AUC as high as a deep learning model, one that learns patterns from data, trained on a training set ten times larger. The authors write that they believe Med-BERT will benefit disease prediction studies with small local training datasets, reduce data collection expenses, and accelerate artificial intelligence aided healthcare.
How does Med-BERT work?
Med-BERT adapts BERT, a language model technique originally developed for text, to structured electronic health records. Researchers pretrained the model on a very large set of records so it learns contextualized embeddings. Those embeddings can then improve models that are fine-tuned, or adjusted, on smaller datasets for specific tasks.
What did the experiments show?
Fine-tuning experiments showed that Med-BERT substantially improves prediction accuracy. It boosted the area under the receiver operating characteristic curve (AUC) by 1.21 to 6.14 percent in two disease prediction tasks from two clinical databases. With small fine-tuning training sets, pretrained Med-BERT boosted AUC by more than 20 percent or matched a deep learning model trained on a training set ten times larger.
What do the authors expect?
The authors state this as their belief: Med-BERT will benefit disease prediction studies with small local training datasets, reduce data collection expenses, and accelerate artificial intelligence aided healthcare. That forward-looking statement is an expectation, not a measured result.
Limits and what's next
This is one study, and the abstract reports no independent replication. The fine-tuning results come from two disease prediction tasks in two clinical databases, so the abstract does not say that the same gains appear in other diseases or settings. The pretraining used a structured electronic health record dataset of 28,490,650 patients, and the abstract does not say how generalizable that population is. The authors' closing statement is a stated belief, not a demonstrated finding, that Med-BERT will benefit small local datasets, reduce data collection expenses, and speed artificial intelligence aided healthcare.
Source: Laila Rasmy et al., “Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction,” npj Digital Medicine, 2021.