Auditing LLM Chemical Priors for Multi-Objective Bayesian Optimization of Chemical Reactions: A Validity, Alignment, and Diversity Benchmark
Year: 2026
Type: preprint
Abstract: Large language models (LLMs) are increasingly proposed as priors for cold-starting Bayesian optimization (BO) of chemical reactions, yet their benefit–risk profile in multi-objective reaction optimization remains poorly characterized. We present a systematic benchmark asking when LLM knowledge helps, is neutral, or hurts multi-objective BO (MOBO). Across seven public reaction benchmarks (6,384 candidate experiments), we compare nine strategies—including LLM-initialized BO, an LLM-only initializer, and established baselines (TSEMO, EDBO+-style scalarization, LLAMBO)—with 20 seeds per configuration (1,120 main-grid plus 360 ablation campaigns), scored by hypervolume completion and its sampling-efficiency integral (AUC), with paired Wilcoxon tests and two-family Benjamini–Hochberg correction against a random-init BO control. Three regimes emerge. On Suzuki–Miyaura, LLM initialization significantly improves sampling efficiency (ΔAUC +2.0 points, q = 0.026; alternative backbone +2.7 points, q = 1.4 × 10⁻⁴). On Buchwald–Hartwig, LLM priors are statistically indistinguishable from random. On a process-development coupling, LLM initialization significantly harms efficiency (ΔAUC −11.1 points, q = 0.0016) despite a 0% out-of-pool rate, whereas the Suzuki benefit coexists with a 45% out-of-pool rate—the prior’s direction, not its syntactic validity, controls the outcome. Diversity-based initialization (farthest point sampling) matches or exceeds LLM initialization wherever LLMs help, and no LLM-derived benefit exceeds what the knowledge-free diversity control already provides. A five-category diagnostic protocol intercepts 100% out-of-pool suggestions on the flow cases at zero experimental cost. LLM priors are not yet a reliable default for multi-objective reaction optimization; we provide a cost-free pre-campaign screening heuristic for deciding when to trust them.
Source: ChemRxiv
Author: Fan Zhang
Language: English
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7214664710