Healthcare & Bio AI
A pathology AI assistant outperformed other multimodal assistants on diagnostic questions and pathologist-preferable open-ended answers in a 2024 study
In a 2024 paper published in Nature, researchers describe PathChat, an AI assistant built to read both pathology images and written questions. The team fine-tuned the whole system on over 456,000 visual-language instructions, which contained 999,202 question-and-answer turns. In the evaluation, PathChat outperformed several other vision-language AI assistants and GPT-4V on multiple-choice diagnostic questions drawn from cases with diverse tissue origins and disease models. On open-ended questions, human expert evaluation found that PathChat's responses were overall more accurate and pathologist-preferable.
What is PathChat?
PathChat is a vision-language generalist AI assistant for human pathology. It combines a vision encoder adapted for pathology with a pretrained large language model, so it can accept images and written language together.
What did the study compare?
The researchers compared PathChat with several multimodal vision-language AI assistants and with GPT-4V, which powers the commercially available general-purpose assistant ChatGPT-4.
What did the testing find?
On multiple-choice diagnostic questions from cases with varied tissue origins and disease models, PathChat recorded the best performance. On open-ended questions, human expert evaluation found that PathChat produced more accurate and pathologist-preferable answers overall.
Frequently asked questions
How much training data did the researchers use?
The abstract reports that the team fine-tuned the whole system on over 456,000 visual-language instructions, made up of 999,202 question-and-answer turns.
Is PathChat in clinical use?
The abstract does not say that PathChat is in clinical use. It says the assistant may potentially find applications in pathology education, research and human-in-the-loop clinical decision-making.
What kind of input does PathChat accept?
It accepts visual and written inputs. The abstract describes it as an interactive vision-language AI copilot that can flexibly handle both visual and natural language inputs.
Limits and what's next
This is one study. The evaluation used multiple-choice diagnostic questions and open-ended questions from cases with diverse tissue origins and disease models. The abstract does not mention independent replication or external validation. The authors describe possible applications in pathology education, research and human-in-the-loop clinical decision-making as potential, not established.
Source: Ming Y. Lu et al., “A multimodal generative AI copilot for human pathology,” Nature, 2024.