Work

Aya in Action: An Investigation of its Abilites in Aspect-Based Sentiment Analysis, Hate Speech Detection, Irony Detection and Question-Answering

Year: 2025

Type: article

Abstract: Low-resource languages face challenges due to limited linguistic resources. In 2024, Aya, a multilingual model supporting 101 languages, was introduced. This study evaluates Aya's performance and the efficacy of a few-shot learning approach in Aspect-Based Sentiment Analysis, Hate Speech Detection, Irony Detection, and Question-Answering using ABSAPT 2022, ToLD-BR, IDPT 2021, and SQUAD v1.1 datasets. Without fine-tuning, Aya demonstrated strong results in QA, achieving a 58.79% Exact Match score, surpassing Portuguese-specific models. However, it struggled in Hate Speech Detection, with an F1-score of 0.64, well below Sabiá-7B's 0.94. ABSA performance improved without neutral examples, but the model faced challenges with complex slang and context-dependent features. These findings highlight Aya's potential in multilingual NLP while demonstrating the capabilities and limitations of few-shot learning as an evaluation strategy for LLMs in low-resource scenarios.

Source: Journal of the Association for Information Systems

Authors: Carvalho, Eduarda Abreu, Lopes, Émerson, da Rocha Junqueira, Júlia, Correa, Ulisses B, Freitas, Larissa

Institutions: Universidade Federal de Pelotas, Universidade Federal do Rio Grande do Sul

Language: English

Topic: Sentiment Analysis and Opinion Mining

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Nvidia, Coordenação de Aperfeiçoamento de Pessoal de Nível Superior, Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul

Awards: G2554800379, G29316538

OpenAlex ID: W6998832241

FWCI:

Cites:

Cited by:

Related to: