Year: 2026
Type: dissertation
Abstract: Deep learning practitioners face numerous manual decisions throughout the model development pipeline, from selecting hyperparameters and optimization algorithms to evaluating model capabilities. These decisions often require extensive expertise and computational resources, creating barriers to effective model development. We present approaches for reducing human intervention across three stages of the deep learning pipeline: hyperparameter selection, optimization, and evaluation. For hyperparameter selection, we explore two complementary approaches. We show that large language models (LLMs) can effectively suggest hyperparameters, matching or outperforming classical approaches on standard benchmarks with limited budgets. For variational autoencoders specifically, we introduce Multi-Rate VAE, which uses hypernetworks to learn optimal parameters across all KL weight values in a single training run, eliminating the need for repeated experiments to explore the rate-distortion tradeoff. For optimization, we present Lookahead, an algorithm that iteratively updates slow and fast weights to reduce the variance and improve the stability of an inner optimizer. Lookahead consistently improves convergence across diverse architectures and tasks while remaining robust to its own hyperparameter choices. For evaluation, we propose Report Cards, which are automatically generated natural language summaries of language model behavior. By using LLMs to analyze model outputs, Report Cards complement traditional metrics, providing insights that would typically require extensive manual analysis. Together, these methods address key bottlenecks in deep learning workflows, working toward the vision of automated machine learning that is accessible to non-experts while reducing computational waste from trial-and-error experimentation.
Source: TSpace (University of Toronto)
Author: Zhang, Michael Ruoqi
Institution: University of Toronto
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204014834