Work

CABY26: Statistics and Analysis of a Cleaned, Re-labeled Cyberbullying Corpus with Agreement-Based Bystander Roles

Year: 2026
Type: preprint
Abstract: This paper introduces CABY26, a fully re-labeled version of the CYBY24 cyberbullying corpus. CABY26 replaces CYBY24's four-level bullying severity scale and biased bystander labels ("instigator", "defender") with a clear binary Aggression / No Aggression judgment and an agreement-based bystander role scheme (agree, disagree, impartial, other), following the remediation framework proposed in the companion guidelines paper. Three annotators labeled the full corpus using a chronological three-stage workflow: they judge the main post alone, then assign a role to each reply, then judge the whole thread. Before analysis, we removed 8 threads with pornographic main posts and 234 replies that were exact duplicates, leaving 2,781 threads and 10,241 replies. Inter-annotator agreement (Fleiss' kappa) falls in the fair range for all three label stages (0.33 to 0.39), bystander stance is linked to thread-level aggression (Cramér's V = 0.144, p < 0.001), and the panel changes its verdict from the isolated main post to the full thread in about 13% of threads, confirming the concept of Change of Mind (CoM) that CYBY24 showcased. We also report the practical challenges the annotator panel faced while labeling, especially the lack of image and video context and the difficulty of judging exactly who a reply's aggression is aimed at, and turn these into concrete recommendations for future annotation projects. We then test whether the relabeling helps models. On a held-out test set of 418 threads and 1,523 replies, fine-tuned BERT and RoBERTa and three prompted Large Language Models (LLMs) all score higher on CABY26 bystander role labels than on CYBY24 labels for the same replies: macro-F1 (0-100) rises from 33.6-43.0 to 46.1-53.0. We release a thread-level 70:15:15 split and benchmark results with the corpus.
Language: English
Open Access status: green
OpenAlex ID: W7233589149
Cites:
Cited by:
Related to: