Year: 2026
Type: preprint
Abstract: Data attribution seeks to quantify how individual training examples shape a model's predictions and underpins problems including data valuation, machine unlearning, and model interpretability. Despite having a long line of work, computationally scalable methods often struggle to predict the effect of removing training data in neural networks due to their non-convex nature. To overcome this challenge, metagradient-based methods such as MAGIC (Ilyas and Engstrom, 2025) differentiate each prediction through the entire training run and compute its exact influence with respect to the training data, but require a separate run for every prediction. To reduce this cost, we cast budgeted attribution as estimating a large influence matrix from a small number of measurements. We show that the measurements most appropriate for recovering this matrix differ from those best suited for attribution itself. We then present two algorithms, MAGE and SPELL, suited for reconstruction and attribution respectively, that run on existing metagradient machinery at no extra cost. Empirical studies demonstrate strong performance over existing baselines across training scales and measurement budgets.
Source: arXiv (Cornell University)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1453181110, Track 4: Advanced CI Coordination Ecosystem: Monitoring and Measurement Services, G3562386289, Track 1: ACCESS Resource Allocations Marketplace and Platform Services (RAMPS), G4010157413, ACO: An Open CI Ecosystem to Advance Scientific Discovery (OpenCI), Track 3: COre National Ecosystem for CyberinfrasTructure (CONECT), G5367127336, G5464919783, Efficient Data Removal Under High-dimensional Asymptotics with Applications to Risk Estimation and Machine Unlearning, Track 2: Customized Multi-tier Assistance, Training, and Computational Help (MATCH) for End User ACCESS to CI
OpenAlex ID: W7213249742