Year: 2024
Type: conference-paper
Abstract: This paper presents the Hallucination Recognition Model for New Experiment Evaluation (HaRMoNEE) team's winning (#1) and #10 submissions for SemEval-2024 Task 6: Sharedtask on Hallucinations and Related Observable Overgeneration Mistakes (SHROOM)'s two subtasks.This task challenged its participants to design systems to detect hallucinations in Large Language Model (LLM) outputs.Team HaRMoNEE proposes two architectures: (1) fine-tuning an off-the-shelf transformer-based model and (2) prompt tuning large-scale Large Language Models (LLMs).One submission from the fine-tuning approach outperformed all other submissions for the model-aware subtask; one submission from the prompt-tuning approach is the 10th-best submission on the leaderboard for the modelagnostic subtask.Our systems also include pre-processing, system-specific tuning, postprocessing, and evaluation.
Authors: Timothy Obiso, Jingxuan Tu, James Pustejovsky
Institution: Brandeis University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: National Science Foundation
OpenAlex ID: W4401042124