HABIBTAZ at SemEval-2026 Task 11: Disentangling Formal Logic from Content via Synthetic Training and Multi-Objective Optimization
Year: 2026
Type: conference-paper
Abstract: While Large Language Models (LLMs) excel in many general NLP tasks, their formal reasoning capabilities are often compromised by content effects, demonstrating a measurable bias towards real-world plausibility.In this paper, we present our system for SemEval-2026 Task 11, which evaluates the ability of models to disentangle formal logic from content across 12 languages with and without distractor premises.We address this challenge using mDeBERTa-v3 networks fine-tuned on a synthetic, rulebased dataset of syllogistic schemes to avoid the semantic noise of LLM-augmented data.To explicitly decouple plausibility from logical structure, our training pipeline employs a multiobjective loss function combining Adaptive Group Distributionally Robust Optimization (DRO), a scheduled differentiable bias penalty, and KL-Divergence consistency regularization.Our system achieved #1 ranks and perfect Ranking Scores (100.0) with 0.00% bias and 100.0%accuracy on Subtask 1 (English), Subtask 2 (Noisy English), and Subtask 3 (Multilingual).On the highly complex Subtask 4 (Noisy Multilingual), the system achieved the 6th rank with 89.06% Accuracy and F1-score, alongside a limited 2.89% Bias and a 37.78 Ranking Score.Our dataset generation engine and codebase are publicly available to facilitate future work on robust logical reasoning.
Institution: Habib University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166888476