Debiasing Logical Fallacy Detection for Real-World Robustness via Counterfactually Augmented Data
Year: 2026
Type: conference-paper
Abstract: Logical fallacy detection models frequently over-flag valid reasoning due to reliance on surface-level spurious correlations.We introduce 703 LLM-generated Counterfactually Augmented Data (CAD) pairs-minimally differentiated valid and fallacious arguments-to debias models through targeted augmentation.Fine-tuning DeBERTa-v3-large on CoCoLoFa augmented with these pairs yields marginal in-distribution improvement (+0.4% F1) but substantial out-of-distribution robustness: 58% relative reduction in false positive rate (64% → 26.7%) on a 300-sample Reddit-sourced evaluation set.While recent LLMs (Llama-3.1-8B,Llama-3.3-70B)achieve high performance under optimized prompts (F1 90-94%), they degrade severely under simple humanlike prompts (F1 63-72%, FPR 54-74%).Our lightweight, prompt-invariant approach achieves competitive robustness (F1 85.9%, FPR 26.7%) across all prompting regimes without prompt engineering, making it stable for production deployment with unpredictable user input.The dataset and model are publicly released.
Author: Navyansh Singh
Institution: International Institute of Information Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7166825268