Excerpt from Towards Explainable, Fine-Grain Online Sexism Classification

15 Apr 2023

---

Methods to identify and classify sexist language online, using machine learning models like Gaussian Naive Bayes, bi-LSTM, and Transformer-based models. It achieved notable results with the RoBERTa-large model, placing in the top 10% for a specific task (SemEval-2023). The study aims to improve explainability in AI models for better understanding complex social issues like online sexism.

Abstract

• Context: Study focuses on identifying and classifying sexist language online, specifically for SemEval 2023 Task 10.
• Methodology: Utilizes various machine learning models like Gaussian Naive Bayes, bi-LSTM, and Transformer-based models for classification tasks.
• Results: Best results with RoBERTa-large model, ranking in the top 10% for subtask B.
• Future Work: Improvements in hierarchical taxonomy classification and enhancing explainability.

1. Introduction

• Background: Increasing negativity of online sexism affecting women and perpetuating inequalities.
• Objective: Developing fine-grain classification to improve interpretability and explainability.
• Demo: An interactive demo platform for showcasing classification and explainability.

2. Related Work

• Prior Efforts: Early efforts using logistic regression for offensive tweet detection. Recent focus on Transformer-based models.
• Comparison: Table 1 compares F1 scores of related works.

3. Methods

• Text Preprocessing: Utilizes Spacy for tasks like lowercasing, lemmatizing, etc.
• Classification Strategies: Includes Per-Level Classification, Beam-Search, and Per-Parent Classification.
• Models: Gaussian Naive Bayes, bi-LSTM, and Transformer models like BERT, RoBERTa.
• Data Augmentation: Techniques like back-translation, synonym replacement for balanced datasets.

4. Experiments

• Dataset: 20,000 labeled entries from Gab and Reddit.
• Baselines: Established using Gaussian Naive Bayes model.
• Evaluation: Focused on macro-averaged F1 score.
• Implementation: Utilized PyTorch Lightning, Google Colab, and Weights & Biases for logging.

5. Results

• Performance: Best results with per-level classification and data augmentation.
• Comparison: Leaderboard rankings and scores compared with competition results.

6. Conclusion

• Findings: Data augmentation and semi-supervised techniques beneficial in fine-grain sexism classification.
• Shortcomings: Issues in tasks with lower taxonomic levels.

7. Future Work

• Focus: Improving methodologies to leverage taxonomic relationships and exploring model explainability.
• Approaches: Investigating label embedding methods and hierarchical masking.

More details can be found in the report

---

Authors: Axel Bogos and Jie Bao, Université de Montréal