Predictive Fit Substituted for Intervention Knowledge: Why a Measure That Forecasts an Outcome May Not Move It
Year: 2026
Type: article
Abstract: This conceptual paper introduces Predictive Fit Substituted for Intervention Knowledge, a structural constraint on what a successful forecast can establish. A measure may track an outcome closely under observed conditions. A proposal to change the outcome must also specify how someone will act on the measure and what evidence supports that action. Different ways of changing the same measure can have different effects. Evidence for a causal claim can come from an intervention or from observations interpreted under defensible causal assumptions. Economist Steven Tadelis recalled that consultants ranked advertisements on searches for eBay’s own name highest for return per dollar. Shoppers who searched for eBay by name, however, could also reach it through an unpaid listing. In 2012, when eBay suspended its brand advertisements on Yahoo and MSN, researchers estimated that it retained 99.5 percent of its search traffic from MSN. A separate experiment in 2013 addressed revenue. Suspending Google brand advertisements in eight randomly selected German states had no measurable short-term effect on eBay’s revenue. The AIM-HIGH trial shows how researchers put the intervention question to a test. Higher HDL cholesterol was associated with lower cardiovascular risk. Among patients with established vascular disease already receiving statin therapy, adding extended-release niacin raised HDL cholesterol. The primary cardiovascular endpoint occurred in 16.4 percent of the niacin group and 16.2 percent of the placebo group. The trial found no added benefit from that treatment strategy in the population studied. Part of the Inquiry and Possibility series within On Human Modeling, the paper places the distinction alongside Nancy Cartwright’s account of causal laws and effective strategies. A team evaluating an AI assistant might find that answers with more citations tend to be more accurate. Whether requiring five citations per answer improves accuracy is a separate question. The team could test that requirement and check the resulting factual claims against their sources.
Author: Cary, Alexandra
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7236577924
