The Effects of Decision-Maker Participation in Training AI Tools for Housing
Year: 2026
Type: preprint
Study design: Clinical Trial
Abstract: This study is an online randomized experiment examining whether participation in training an AI decision-support tool affects subsequent human decision-making in a housing policy task. Participants complete a task about prioritizing vacant or abandoned houses for demolition using street-view images and AI recommendations. Participants are randomly assigned to a control condition or a treatment condition. In both conditions, participants first complete stage one, in which they make pairwise property decisions with street-view images, AI recommendations, and yes/no labels indicating whether each property has roof damage, broken windows, and wall damage. They then complete stage two on a new set of property pairs, again with AI recommendations but without the yes/no feature labels. In the control condition, stage one is framed as practice for stage two. In the treatment condition, participants are told before stage one that the stage one task is part of training the AI model and that their choices will be used to update the model before stage two. Within each stage, property pairs include baseline-model pairs and trained-model pairs; only trained-model pairs can have their stage two AI error distribution changed for treatment participants. The primary goal is to estimate whether decision-maker participation in AI model training changes stage two decision quality and reliance on AI recommendations. The three primary outcomes are overall stage two accuracy, overall stage two reliance on the AI recommendation, and stage two accuracy on baseline-model pairs. We will test whether treatment and control participants differ on each of these outcomes.
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7161566564