Work

Closing the Gaps on Inscrutability: Tackling Challenges with Knowledge Integration during AI development

Year: 2025

Type: article

Abstract: The development of complex artificial intelligence (AI) systems presents a compelling knowledge integration challenge to organisations. As the organisations strive to integrate complex domain knowledge into algorithmic models, they also have to arm domain experts with the technical understanding of how such models work so they can be used responsibly. The inscrutability of AI technology – stemming from challenges related to both the technical explainability of the models as well as their social interpretability – makes knowledge integration particularly challenging by creating and deepening knowledge gaps between the AI model, its human users and domain reality. To increase understanding of how such knowledge gaps can be addressed in AI development, this study reports on three qualitative case studies on AI projects faced where inscrutability needed to be managed. Building on the gap model (Kayande et al., 2009), we identify three sociotechnical mechanisms for addressing knowledge gaps related to AI inscrutability and thus facilitating organisational learning. Our work provides contributions to both theory and practice.

Source: AJIS. Australasian journal of information systems/AJIS. Australian journal of information systems/Australian journal of information systems

DOI: 10.3127/ajis.v29.5567

Authors: Tapani Rinta-Kahila, Ida Someh, Ali Darvishi, Reihaneh Bidar, Marta Indulska

Institution: The University of Queensland

Language: English

Topic: Explainable Artificial Intelligence (XAI)

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funder: Australian Research Council

Awards: Maintaining Human Expertise in an AI-driven World, ARC Training Centre for Information Resilience