AI xPress

AI and computer science research.

Back to Foragentis

AI & Society

Models that classify hate speech well can still explain their decisions poorly, and training with human rationales was linked to less bias, a 2021 paper reports

AI & Society November 16, 2026

A delivery rider on a bike in city rain

In a 2021 paper in Proceedings of the AAAI Conference on Artificial Intelligence, researchers at Indian Institute of Technology Kharagpur and Universität Hamburg introduced a dataset called HateXplain. They found that models which sorted social media posts into hate, offensive, or normal could perform well on classification yet still score low on explanation plausibility and faithfulness. Plausibility measures how convincing an explanation seems, and faithfulness measures how closely the explanation tracks the model's actual behavior. Models trained with human rationales, the parts of a post that a person cited as the basis for a label, showed less unintended bias toward the communities targeted in the posts.

What did the dataset add?

The dataset labels each social media post from three angles: the usual category of hate, offensive, or normal, the target community, and the human rationale. The target community is the group the post attacks or disparages. The rationale marks the portions of the post that an annotator used to decide the label. This design lets researchers ask whether a model that gets the label right also points to the right reason.

What did the study find?

Even models that scored high on classification did not score high on explanation plausibility and faithfulness. In other words, being accurate about the label did not mean the model could explain its decision in a way that matched human judgment or reflected what it actually did. The researchers also observed that models trained with human rationales performed better at reducing unintended bias toward the communities the posts targeted. The abstract describes this as a performance difference in one benchmark, not as proof that the rationales caused the lower bias.

What is a rationale, and why did it matter?

A rationale is the portion of a post that a human annotator marked as the basis for a label of hate, offensive, or normal. The researchers used human rationales both to judge model explanations and as extra guidance during training. Models that received that guidance performed better at lowering unintended bias toward target communities.

Limits and what's next

This study examined social media posts in a benchmark dataset and tested existing detection models. The abstract does not give the number of posts, the platforms they came from, the languages covered, or the time period. The design is a dataset annotation and model benchmark, not an outside validation. This is one study, and the abstract reports no independent replication of the bias result. Because the authors made the code and dataset public, other researchers have a way to test how far the findings hold beyond this benchmark.

Source: Binny Mathew et al., “HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection,” Proceedings of the AAAI Conference on Artificial Intelligence, 2021.