Year: 2024
Type: conference-paper
Abstract: Sample diversity depends on the task; within mathematics, precision and determinism are paramount, while storytelling thrives on creativity and surprise.This paper presents a simple self-regulating approach where we adjust sample diversity inference parameters dynamically based on the input prompt-in contrast to existing methods that require expensive and inflexible setups, or maintain static values during inference.Capturing a broad spectrum of sample diversities can be formulated as a straightforward self-supervised inference task, which we find significantly improves the quality of responses generically without model retraining or fine-tuning.In particular, our method demonstrates significant improvement in all supercategories of the MMLU multitask benchmark (GPT-3.5:+4.4%, GPT-4: +1.5%), which captures a large variety of difficult tasks covering STEM, the humanities and social sciences.
Institution: Durham University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4401044145