AI & Society
XLM-Roberta reached 0.841 accuracy in detecting cyberbullying in Bangla and Chittagonian social media texts, a study reports
In a 2024 paper in Electronics, researchers collected over 5000 Bangla and Chittagonian text samples from social media. They compared traditional machine learning, deep learning, hybrid and transformer models for detecting cyberbullying. The transformer model XLM-Roberta had the highest reported accuracy, 0.841. Deep learning methods in general beat traditional machine learning methods in this comparison.
What did the study measure?
The study measured how reliably different models could identify cyberbullying content in two low-resource languages. The authors describe Bangla as low-resource and Chittagonian as extremely low-resource. They used Krippendorff’s alpha and Cohen’s kappa to measure the reliability of the dataset annotations.
How did the models perform?
Traditional machine learning methods reached accuracies between 0.63 and 0.711. The support vector machine, or SVM, was the best performer in that group. Ensemble models produced similar results. Voting reached 0.72, Bagging reached 0.70, and Boosting reached 0.69.
Deep learning models, including CNN, reached accuracies from 0.69 to 0.811. The authors report that CNN showed the highest accuracy in that group.
Hybrid networks also reported higher accuracies than traditional machine learning. The most complex hybrid, (CNN+LSTM)+BiLSTM, reached 0.82 accuracy. Other hybrid accuracies ranged from 0.78 to 0.804.
Transformer models reached the highest scores in the study. XLM-Roberta reached 0.841. Bangla BERT reached 0.822, Multilingual BERT reached 0.821, BERT reached 0.82, and Bangla ELECTRA reached 0.785.
Why does this matter for low-resource languages?
The authors argue that transformer models can work around the language dependence problem that affects conventional transfer learning methods. They say their findings suggest hybrid approaches and transformer-based embeddings can help detect cyberbullying across online platforms in different linguistic contexts.
FAQ
Which model reached the highest accuracy?
XLM-Roberta, a transformer-based multilingual model, reached 0.841 accuracy. That was the highest accuracy reported in the study.
How much better were deep learning models than traditional machine learning methods?
Traditional machine learning methods reached accuracies from 0.63 to 0.711. Deep learning models, especially CNN, reached accuracies from 0.69 to 0.811. The authors report that deep learning outperformed traditional machine learning approaches.
What languages were studied?
The study used Bangla and Chittagonian text samples from social media. The authors describe both as low-resource languages, and Chittagonian as extremely low-resource.
Limits and what's next
This study tested models on one collected dataset of over 5000 Bangla and Chittagonian social media text samples. The abstract reports no independent replication of these results. The abstract does not say whether the models were tested on text from other social platforms or on newer language data outside the collected sample. The findings describe performance on the languages and data studied. The abstract suggests that hybrid approaches and transformer-based embeddings could help on online platforms, but it reports no external validation.
Source: Tanjim Mahmud et al., “Exhaustive Study into Machine Learning and Deep Learning Methods for Multilingual Cyberbullying Detection in Bangla and Chittagonian Texts,” Electronics, 2024.