Year: 2026
Type: dissertation
Abstract: Machine learning approaches enrich the development of mathematical models of complex phenomena from high-dimensional, noisy data. However, since these model frameworks are so flexible, there is no guarantee that the resulting predictions will generalize to new data. Porting optimization-based tools for fitting traditional, parametric statistical models to training machine learning models consistently yields spurious and overspecified outcomes, suggesting that these complex models may possess a different nature than their predecessors that requires investigation in order to design effective training protocols. This thesis introduces simmering, a physics-based alternative to optimization. Inspired by the effectiveness of statistical mechanics techniques at elucidating macroscopic behaviour in high-dimensional physical systems, simmering recasts optimization problems into an analogous system of one-dimensional, classical “particles” with an interaction potential specified by the optimization objective. To construct a model, simmering employs a molecular dynamics thermostat algorithm to inject entropy into parameter sampling and construct canonical ensembles of parameter-particles. The resulting ensembles facilitate probabilistic interpretations of model-outcome related observables, and achieve superior generalizability when aggregated. In this thesis, I apply simmering to three classes of machine learning problems. First, I show that simmering can “undo” overfitting, produce generalizable outcomes from the outset in regression and classification, and outperform other overfitting mitigation approaches in classification and natural language processing tasks. Next, I introduce the Entropic Autoencoder, a probabilistic, neural-network based dimensionality reduction tool that employs simmering to induce an entropic prior on learned low-dimensional representations, thereby avoiding the generalizability pitfalls of restricting representations to specific variational families. Finally, I apply simmering to construct “causal atlases”, which are maximum-entropy ensembles of Bayesian networks that provide information-theoretically minimally-biased representations of uncertainty and causal structure. In all of these contexts, my physics-based framework excels because, rather than assuming the underlying structure of the phenomenon, it injects landscape-geometry specific noise via entropy maximization and allows the appropriate model to emerge from the ensemble. Moreover, simmering's effectiveness indicates that machine “learning” is an emergent phenomenon and that my entropic approaches uniquely leverage this emergence to construct generalizable models.
Author: Babayan, Irina
Institution: Queen's University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7208233562