AI & Society
A hierarchical LSTM model reached 97.49% accuracy at detecting hatred and trolling in Hindi-English social media text
In a 2021 paper in Complex & Intelligent Systems, researchers described a method for classifying hateful and trolling content in code-mixed social media text. Code-mixed means Hindi and English appear together in the same post. The model, a hierarchical long short-term memory (LSTM) network, combined word embeddings and character embeddings, which are numerical patterns for whole words and for the letters inside them. The authors trained the model to recognize hateful words and to infer the user's intent behind a word. In the paper's comparison, the hierarchical LSTM model reached 97.49% accuracy.
What kind of text did the model study?
The study used Hindi-English code-mixed social media data. The authors ordered the data into Hindi, English, and hatred labels for classification. The authors trained the model to identify hatred and trolling words in social media content.
How did the model compare with standard classifiers?
The authors evaluated the hierarchical LSTM model against BLSTM, CRF, LR, SVM, Random Forest and Decision Tree models. In that evaluation, the authors report 97.49% accuracy for the hierarchical LSTM model, especially when the social media data contained hatred and trolling words. The abstract does not give the exact scores for the other classifiers.
What does hierarchical LSTM mean?
An LSTM is a neural network designed to process sequences, such as words in a sentence. The abstract describes the hierarchical LSTM as a learning architecture inspired by neural learning models. The paper uses this design to detect hatred and trolling in mixed Hindi-English posts.
What data did the paper use?
The paper used Hindi-English code-mixed social media text. The authors ordered the data into Hindi, English, and hatred labels for classification. The abstract does not name the dataset or report its size.
Limits and what's next
The evaluation covered Hindi-English code-mixed social media text, so the reported 97.49% accuracy belongs to that setting. The abstract does not name the dataset, report its size, or say how the researchers collected the data. The authors benchmarked the model against standard classifiers. The abstract does not report an independent replication. This is one study. The abstract does not say what the next step would be.
Source: Shashi Shekhar et al., “Hatred and trolling detection transliteration framework using hierarchical LSTM in code-mixed social media text,” Complex & Intelligent Systems, 2021.