Takoyaki at SemEval-2026 Task 3: Ensembling LLM Predictions using Demonstration Retrieval for Dimensional Aspect-based Sentiment Analysis
Year: 2026
Type: conference-paper
Abstract: This paper describes our system for SemEval-2026 Task 3 (DimABSA).We participate in Subtask 2 (DimASTE), which requires extracting triplets of aspect term, opinion term, and valence-arousal scores from review sentences, and Subtask 3 (DimASQP), which additionally requires aspect category classification to form quadruplets.Our proposed system consists of a multi-step pipeline: (1) retrievalbased in-context learning using BM25 to select relevant demonstrations for LLM inference, (2) agreement-based ensemble combining LLM predictions from multiple retrieval variants, and, for a subset of datasets, (3) error-pattern correction refining uncertain predictions using correction rule sets based on training data.Retrieval-based ICL and the agreement-based ensemble show consistent improvements across languages and domains.Error-pattern correction yields further improvement for the Japanese dataset.To further investigate output quality beyond automated evaluation metrics, we conducted human evaluation.The results suggest that LLM-based labeling achieves higher agreement with gold labels than human annotators, and additionally indicate a discrepancy between automated scores and practical output quality.
Authors: Kosuke Yamada, Sho Takase, Ryosuke Kohita
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166862855