AI & Society
Language models trained on broad text carry political leanings that reinforce polarization and skew hate speech and misinformation detection
In a 2023 paper, researchers found that language models, software trained on large text collections, carry measurable political leanings. Those leanings reinforce the polarization already present in the training text. They also carry social biases into models built to spot hate speech and misinformation.
What did the researchers measure?
The study measured political bias in language models along social and economic axes. It also measured fairness in downstream natural language processing models, meaning models built for specific tasks. The researchers focused on hate speech detection and misinformation detection. The language models had first been trained on broad text sources, including news, discussion forums, books, and online encyclopedias. The authors note that a significant portion of this data includes opinions and perspectives.
What did they find?
The language models showed political leanings. Those leanings reinforced the polarization present in the training text. The social biases then propagated into hate speech predictions and misinformation detectors. The authors describe these detection tasks as high-stakes tasks with social consequences.
FAQ
How did political bias show up in the models?
The researchers found political leanings that reinforced polarization already present in the training text. The leanings appeared along social and economic axes.
Why does this matter for hate speech and misinformation detection?
Hate speech and misinformation detectors are built on top of these language models. When the underlying model carries social bias, that bias can propagate into the detector's predictions. The authors say this creates fairness concerns for high-stakes tasks.
Limits and what's next
The abstract does not state specific limitations. The authors discuss implications for natural language processing research and propose future directions to reduce unfairness, but the abstract does not spell out those directions.
Source: Shangbin Feng et al., “From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models,” 2023.