EViSE: An Efficient Framework for Multi-Criteria Summarization Evaluation
Year: 2026
Type: preprint
Abstract: Reliable summarization evaluation requires criterion-sensitive scoring, computationally practical inference, and useful behavior during model optimization. We propose EViSE, an efficient multi-criteria summarization evaluation framework with a criterion-wise, group-balanced cardinal–ordinal objective coupling score regression with within-source ranking. The effectiveness of EViSE is demonstrated in several aspects: The framework provides a general formulation for grouped multi-criteria learning, with objective-level properties established through mathematical analysis independently of the encoder implementation. EViSE achieves criterion-wise agreement with human judgments in multi-criteria summarization evaluation while efficiently predicting Faithfulness, Coherence, and Relevance in one reference-free encoder pass. A matched WikiLingua application demonstrates downstream reuse as a frozen reward evaluator under group relative policy optimization (GRPO), with similar human-reviewed quality after evaluator substitution. Using a corpus of 80,856 human-reviewed Vietnamese document–summary instances, we evaluate two instantiations of the framework. On held-out News, Overall Spearman correlations are 0.737 for EViSE-ViDeBERTa and 0.806 for EViSE-mmBERT. Both retain substantial agreement under domain shift to information technology textbooks. The compact instantiation achieves 55.2 ms/pair latency on a matched Tesla T4 profile. The formulation can be adapted to other languages using suitable encoders and criterion-labeled source–summary groups.
Source: Research Square
Institution: VNU University of Engineering and Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225709743
Cites:
Cited by:
Related to: