AI & Society
With SMOTE-balanced data and TF-IDF features, a two-model voting classifier reached 0.97 accuracy at identifying toxic social media comments
In a 2021 study published in IEEE Access, researchers tested an ensemble called the regression vector voting classifier (RVVC). The ensemble combines a logistic regression classifier and a support vector classifier under soft voting criteria. Soft voting means each model's confidence score contributes to the final label, whether a comment is classified as toxic or not toxic. On data balanced with SMOTE and represented with TF-IDF word features, RVVC reached 0.97 accuracy and did better than every other individual model tested. Toxic comments are rude, disrespectful, or hateful comments on social media and microblogging sites.
What problem did the study address?
Social media platforms host discussions where fights can produce rude, disrespectful, and hateful comments. The paper describes identifying these toxic comments as an important task for social media platforms.
How does the RVVC ensemble work?
RVVC combines a logistic regression classifier and a support vector classifier under soft voting criteria. Soft voting means the two models' confidence scores are combined to choose the final label, rather than simply counting one vote per model.
What did the researchers test?
The researchers ran experiments on an imbalanced dataset and on a balanced dataset. They used SMOTE on the imbalanced dataset to create balance. They also compared two word representation methods, TF-IDF and bag-of-words, and compared RVVC with several machine learning classifiers.
What did the results show?
RVVC did better than all other individual models when TF-IDF features were used with the SMOTE-balanced dataset. That configuration reached an accuracy of 0.97.
What should a reader know about the methods?
What is SMOTE?
SMOTE stands for synthetic minority oversampling technique. It creates new examples of the minority class so the training data is less imbalanced.
What do TF-IDF and bag-of-words do?
Bag-of-words counts how often each word appears in a text. TF-IDF, or term frequency-inverse document frequency, weights words by how often they appear in a document and how rare they are across the set of documents.
Limits and what's next
This is one study. The abstract reports no independent replication. The abstract does not name the specific dataset, the social media platforms, or the number of comments used. It also does not name the individual machine learning classifiers used for comparison. The abstract reports accuracy but not the precision, recall, or F1-score values from the comparison. The abstract does not describe next steps.
Source: Vaibhav Rupapara et al., “Impact of SMOTE on Imbalanced Text Features for Toxic Comments Classification Using RVVC Model,” IEEE Access, 2021.