HyperBench-7: A Multimodal Metadata for Hyperparameter Selection, Model Benchmarking, and AutoML Agent Evaluation
Year: 2026
Type: conference-paper
Abstract: Selecting hyperparameters in machine learning presents significant challenges due to the requirement of numerous trial-and-error runs or expensive optimizer-based methods. This paper presents HyperBench-7, a comprehensive metadata dataset comprising seven modalities (Image, Text, Tabular, Graph, Time-Series, Audio, and Video) and their optimal hyperparameter configurations. The collection includes 2,000 configurations, each with standardized metadata like dataset size and class count, enabling detailed quantitative investigation of design trends. Exploratory Data Analysis shows unique patterns for each modality. The dataset provides “safe default” hyperparameter configurations tailored to each modality, offering reliable baselines for new datasets. In contrast to previous benchmarks, we combine multi-modal data with specific hyper-parameter settings, providing a new level of model performance understanding across different datasets. The dataset serves as a valuable, reusable resource for meta-learning, AutoML and research into hyperparameter performance dynamics. Further, we evaluated six agent types for predicting model, batch size, and evaluation metrics. Among the six agents evaluated for predicting model, batch size, and evaluation metrics, the LLM-based agent performed best, achieving an overall accuracy of 66.7% approximately. By combining it with these agent types, we can advance AutoML research with multi-modal hyperparameter dynamics and well-founded agentic recommendations. This joint approach provides “safe default” hyperparameters that serve as good benchmarks for any new dataset.
Institutions: R. P. Shaha University, University of Essex
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7204146778