Language & LLMs
Language models often miss relevant information in the middle of a long input
In a 2024 paper published in Transactions of the Association for Computational Linguistics, a peer-reviewed journal, researchers at Stanford University and the University of California, Berkeley studied how language models use long input passages. Language models are software trained to predict text. The researchers tested two tasks that require finding relevant information inside the input: answering questions across several documents and retrieving stored key-value pairs.
They found that performance could fall sharply when relevant information appeared in the middle of a long input. Performance was often highest when the relevant information sat near the beginning or the end. The drop in the middle showed up even for models designed to handle long contexts.
What did the study measure?
The study measured how well language models found relevant information in long input contexts. In multi-document question answering, a model must find an answer across a set of documents. In key-value retrieval, a model must return the value tied to a key in the input.
Where did performance change?
Performance often depended on where the relevant information was placed. Models tended to do best when that information was at the beginning or the end. Their performance fell significantly when they had to reach information in the middle of a long context.
Common questions
What is a long context?
A long context is a stretch of text or stored information that a language model receives at once as input. The study examines whether models can find the relevant piece when that input is long.
Did the study name individual models?
No. The abstract reports tests across language models, including models designed for long contexts, but it does not name individual models or give counts.
Does this mean models always fail in the middle?
No. The abstract reports a tendency, not a constant outcome. Performance often fell and fell significantly in the middle, but the abstract does not claim failure in every case.
Limits and what's next
The abstract describes one study of language models on two retrieval-style tasks: multi-document question answering and key-value retrieval. It reports no independent replication. The observed pattern is a position effect, not a causal finding. The authors describe their contribution as a better understanding of how language models use input context and new evaluation protocols for future long-context models.
Source: Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics, 2024.