Work

Δ-model correction of foundation model based on the model’s own understanding

Year: 2025
Type: article
Abstract: Foundation models of interatomic potentials, the so called universal potentials, may require fine-tuning or residual corrections when applied to specific subclasses of materials. In the present work, we demonstrate how such an augmentation can be accomplished via Δ-learning based on the representation already embedded in the universal potentials. The Δ-model introduced is a Gaussian Process Regression (GPR) model, and various types of aggregation (global, species-separated, and atomic) of the representation vector are discussed. Employing a specific universal potential, CHGNet [Deng et al., Nat. Mach. Intell. 5, 1031 (2023)], in a global structure optimization setting, we find that it correctly describes the energetics of the "8" Cu oxide, which is an ultra-thin oxide film on Cu(111). The universal potential model even predicts a more favorable structure compared with that discussed in recent density functional theory-based literature. Moving to sulfur adatom overlayers on Cu(111), Ag(111), and Au(111), the CHGNet model, however, requires corrections. We demonstrate that these are efficiently provided via the GPR-based Δ-model formulated on CHGNet's own internal atomic embedding representation. The need for corrections is tracked to the scarcity of metal-sulfur atomic environments in the materials project database that CHGNet is trained on, leading to an overreliance on sulfur-sulfur atomic environments. Other universal potentials trained on the same data, MACE-MP0, SevenNet-0, and ORB-v2-only-MPtrj, show a similar behavior but with varying degrees of error, demonstrating the general need for augmentation schemes for universal potential models.
Institution: Aarhus University
Language: English
Open Access status: green
OpenAlex ID: W4410212317
FWCI: 1.654
Cites: 50
Cited by: 8
Related to: 10