Work

Bias Bluff Busters at FIGNEWS 2024 Shared Task: Developing Guidelines to Make Bias Conscious

Year: 2024

Type: conference-paper

Abstract: This paper details our participation in the FIGNEWS-2024 shared task on bias and propaganda annotation in Gaza conflict news.Our objectives were to develop robust guidelines and annotate a substantial dataset to enhance bias detection.We focused on creating guidelines that prompt annotators to be aware of their biases and emphasise the need for a diverse annotator pool, to ensure that high inter-annotator agreement (IAA) reflects genuine consensus rather than homogeneity.We iteratively refined our guidelines and included detailed examples for clarity.We also explored the integration of ChatGPT as an annotator to support consistency.Our findings include the challenges in eliciting annotation consistency without embedding bias in the guidelines themselves and the importance of fostering annotator awareness of their own biases.This work provides insights into the complexities of subjective bias annotation and offers well-crafted guidelines for the field.

Source: Arabic Natural Language Processing Workshop (WANLP)

DOI: 10.18653/v1/2024.arabicnlp-1.62

Authors: Jasmin Heierli, Silvia Pareti, Serena Pareti, Tatiana Lando

Institutions: ZHAW Zurich University of Applied Sciences, Università Cattolica del Sacro Cuore

Language: English

Topic: Sentiment Analysis and Opinion Mining

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W4402671171

FWCI:

Cites:

Cited by:

Related to: 10