Work

Effect of AI Performance, Risk Perception, and Trust on Human Dependence in Deepfake Detection AI System

Year: 2025

Type: article

Abstract: Synthetic images, audio, and video can now be generated and edited by Artificial Intelligence (AI). In particular, the malicious use of synthetic data has raised concerns about potential harms to cybersecurity, personal privacy, and public trust. Although AI-based detection tools exist to help identify synthetic content, their limitations often lead to user mistrust and confusion between real and fake content. This study examines the role of AI performance in influencing human trust and decision making in synthetic data identification. Through an online human subject experiment involving 400 participants, we examined how varying AI performance impacts human trust and dependence on AI in deepfake detection. Our findings indicate how participants calibrate their dependence on AI based on their perceived risk and the prediction results provided by AI. These insights contribute to the development of transparent and explainable AI systems that better support everyday users in mitigating the harms of synthetic media.

Source: Proceedings of the ACM on Human-Computer Interaction

DOI: 10.1145/3757588

Authors: Yingfan Zhou, Ester Chen, Manasa Pisipati, Aiping Xiong, Sarah Rajtmajer

Institutions: Pennsylvania State University, Rochester Institute of Technology

Language: English

Topic: Misinformation and Its Impacts

Subfield: Sociology and Political Science

Field: Social Sciences

Domain: Social Sciences

Open Access status: bronze

Funder: National Science Foundation

Awards: SaTC: CORE: Small: Toward Privacy Equity through Contextual Understanding of Self-Disclosure, CAREER: Enabling Human Situation Awareness across Physical and Cyber Spaces for Vehicle-to-Everything Communication Security and Driving Safety