Target Factorization: Loss Geometry and Component Learning Dynamics
Year: 2026
Type: preprint
Abstract: A class label can combine attributes, such as shape and color. We ask whether predicting these attributes separately makes each easier to learn. For cross-entropy, additive component scores remove an exact cross-component curvature term from the loss. This geometric separation does not guarantee faster learning. In a binary model, learned interactions slow additive-coordinate growth while accelerating predictive confidence: locally at unequal positive rates and at every confidence level above chance at unit rates. In a matched Boolean experiment with 12 initializations, a component reaches an accuracy threshold later when paired with an easier companion target. Allowing interactions reduces this delay from 24.00 to 14.58 updates (paired difference −9.42; exploratory 95% bootstrap interval [−14.33, −4.33]). Integrated prediction loss and error show the same direction without an accuracy threshold. Predictive dependence and backbone gradient alignment do not establish a causal explanation. A parity-target grid retains positive joint-versus-isolated training contrasts under exact output factorization, although its heads are not fully matched. Separating predictions therefore need not make their learning independent. Comparisons of output encodings should measure component-level predictions under explicit training controls.
Source: Preprints.org
Author: Yunguo Yu
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7234454158
