“Why Do I Care What’s Similar?” Probing Challenges in AI-Assisted Child Welfare Decision-Making through Worker-AI Interface Design Concepts
Year: 2022
Type: conference-paper
Abstract: Data-driven AI systems are increasingly used to augment human decision-making in complex, social contexts, such as social work or legal practice. Yet, most existing design knowledge regarding how to best support AI-augmented decision-making comes from studies in comparatively well-defined settings. In this paper, we present findings from design interviews with 12 social workers who use an algorithmic decision support tool (ADS) to assist their day-to-day child maltreatment screening decisions. We generated a range of design concepts, each envisioning different ways of redesigning or augmenting the ADS interface. Overall, workers desired ways to understand the risk score and incorporate contextual knowledge, which move beyond existing notions of AI interpretability. Conversations around our design concepts also surfaced more fundamental concerns around the assumptions underlying statistical prediction, such as inference based on similar historical cases and statistical notions of uncertainty. Based on our findings, we discuss how ADS may be better designed to support the roles of human decision-makers in social decision-making contexts.
Institutions: Carnegie Mellon University, University of Minnesota
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
Awards: EAGER:AI-DCL:Capture, Explain and Negotiate the Inherent Trade-offs in Machine Learning Algorithms, SCC-IRG Track 1: Empowering and Enhancing Workers Through Building A Community-Centered Gig Economy, FAI: Advancing Fairness in AI with Human-Algorithm Collaborations, CHS:Small: Incorporating and Balancing Stakeholder Values in Algorithm Design
OpenAlex ID: W4281939280