Year: 2026
Type: conference-paper
Abstract: Scientific writing is an expert-domain task that demands deep domain knowledge, task-specific requirements and reasoning capabilities that leverage the domain knowledge to satisfy the task specifications.While scientific text generation has been widely studied, its evaluation remains a challenging and open problem.It is critical to develop models that can be reliably deployed for evaluating diverse openended scientific writing tasks while adhering to their distinct requirements.However, existing LLM-based judges and reward models are primarily optimized for general-purpose benchmarks with fixed scoring rubrics and evaluation criteria.Consequently, they often fail to reason over sparse knowledge of scientific domains when interpreting task-dependent and multi-faceted criteria.Moreover, fine-tuning for each individual task is costly and impractical for low-resource settings.To bridge these gaps, we propose cost-efficient, open-source reward models tailored for scientific writing evaluation.We introduce a two-stage training framework that initially optimizes scientific evaluation preferences and then refines reasoning capabilities.Our multi-aspect evaluation design and joint training across diverse tasks enable fine-grained assessment and robustness to dynamic criteria and scoring rubrics.Experimental analysis shows that our training regime strongly improves LLM-based scientific writing evaluation.Our models generalize effectively across tasks and to previously unseen scientific writing evaluation settings, allowing a single trained evaluator to be reused without task-specific retraining.We make our code 1 and data 2 publicly available.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166873827