AI & Society
Racial bias in a Twitter hate speech classifier fell after a reweighting step, a 2020 study finds
In a 2020 paper in PLoS ONE, researchers report that a trained language model labeled tweets written in African-American English as racist, sexist, hateful or offensive more often than tweets written in Standard American English. After the researchers added a reweighting step during training, that racial gap shrank significantly. The model began with BERT, a pre-trained language model, and the team fine-tuned it on two public Twitter datasets annotated for racism, sexism, hate or offensive content.
What did the study find?
The researchers fine-tuned the BERT model on two public Twitter datasets. Those datasets carried labels for racism, sexism, hate or offensive content. They then tested the trained classifiers on two additional Twitter datasets, one with tweets written in African-American English and one with tweets written in Standard American English. The classifiers assigned the African-American English tweets to negative categories more often than the Standard American English tweets. That pattern showed systematic racial bias in the trained classifiers. After the researchers added the bias-alleviation step, the bias reduced significantly.
How did the bias-reduction step work?
The researchers used an existing regularization method, a technique that changes how much influence each training example has. The method reweighted training samples so that wording closely tied to class labels had less effect. The team then fine-tuned the pre-trained BERT model on the reweighted samples.
FAQ
How did the researchers detect bias?
They compared the trained classifier's labels on tweets written in African-American English with its labels on tweets written in Standard American English.
Did the fix remove the bias entirely?
The abstract reports that racial bias reduced significantly. It does not say the bias disappeared.
What model did the researchers use?
They used BERT, a pre-trained language model, and fine-tuned it for hate speech detection.
Limits and what's next
This is one study, and the abstract reports no independent replication. The work used text datasets from Twitter. The researchers fine-tuned BERT on two public Twitter datasets annotated for racism, sexism, hate or offensive content. They then tested the classifier on two further Twitter datasets, one written in African-American English and one written in Standard American English. The design is a computational evaluation of language classifiers, so the findings describe model behavior on those datasets. The authors describe the work as a possible first step toward debiasing hate speech and abusive language detection systems.
Source: Marzieh Mozafari et al., “Hate speech detection and racial bias mitigation in social media based on BERT model,” PLoS ONE, 2020.