RCI — the Revision Contingency Index
Year: 2026
Type: preprint
Abstract: RCI = RAIR + RSR − 1 turns the two standard rates of appropriate reliance on advice into a single number: 0 when revising a decision is unrelated to whether the advice was right, +1 when an agent revises exactly when it should, −1 for the reverse. Reliance is currently reported as a pair of rates (RAIR, RSR; Schemmer et al., 2023), which has no chance level and cannot rank agents. Their sum minus one is the statistic known as Youden’s J and ΔP; this note names it for reliance on advice, states when it describes an agent, and shows that it must be computed per phase when behaviour changes over a task. Because it needs only four counts and does not depend on how often the advice is right, RCI is comparable across people, language models and AI agents. Applied to published studies, it reveals groups that revise against the evidence; applied to nine language models, it shows a responsive phase followed by one in which 8 of 9 models score exactly 0. A reference implementation is included.
Author: Kovalchuk, Anna
Language: English
Subfield: Human Factors and Ergonomics
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7234298897
