AI xPress

AI and computer science research.

Back to Foragentis

AI & Society

Large language models outperformed common toxicity detectors and GPT-3.5 moderated Reddit communities with 64% median accuracy in a 2024 study

AI & Society November 3, 2026

A job centre waiting area

In a 2024 study published in the Proceedings of the International AAAI Conference on Web and Social Media, researchers at the University of California San Diego and Stanford University tested large language models on two kinds of content moderation. The models did better than widely used toxicity classifiers at spotting toxic content. For rule-based moderation, GPT-3.5 reached a median accuracy of 64% and a median precision of 83% when the researchers prompted it with rules from 95 Reddit communities. Bigger models added only marginal benefit on the toxicity task.

What did the study test?

The study tested two common moderation jobs. One was rule-based community moderation. The other was toxic content detection. The researchers benchmarked the models on both tasks.

How well did GPT-3.5 handle Reddit rules?

The researchers created 95 versions of GPT-3.5. They prompted each version with the rules of one Reddit subcommunity. Across those communities, GPT-3.5 reached a median accuracy of 64% and a median precision of 83%. The abstract describes that result as effective for many communities.

How did language models compare on toxicity detection?

The study evaluated GPT-3, GPT-3.5, GPT-4, Gemini Pro, and LLAMA 2. These language models significantly outperformed toxicity classifiers that are widespread today.

Did bigger models help?

Bigger models did not help much. Increases in model size produced only marginal benefit on toxicity detection. The researchers suggest that language models may be approaching a performance plateau on this task.

Common questions

Which models did the study evaluate?

For toxicity detection, the study evaluated GPT-3, GPT-3.5, GPT-4, Gemini Pro, and LLAMA 2. For Reddit rule moderation, it used GPT-3.5 with rules from 95 subcommunities.

What were the two moderation tasks?

The tasks were rule-based community moderation and toxic content detection.

What fields does this work touch?

The abstract lists computer science, linguistics, and philosophy among its keywords.

Limits and what's next

The abstract describes one benchmark study. It tested Reddit rule moderation with rules from 95 subcommunities and toxicity detection across several large language models, but it does not describe the toxicity test data in detail. The abstract reports no independent replication. The authors state that larger models added only marginal benefit on toxicity detection, so they suggest a possible performance plateau. The abstract ends by outlining areas for future work on language models and content moderation.

Source: Deepak Kumar et al., “Watch Your Language: Investigating Content Moderation with Large Language Models,” Proceedings of the International AAAI Conference on Web and Social Media, 2024.