Validation of the QAMAI tool in Italian for the evaluation AI-generated health information in head and neck surgery
Year: 2026
Type: article
Abstract: BACKGROUND: This study aimed to validate the Italian version of the Quality Assessment of Medical Artificial Intelligence (IT-QAMAI) tool, designed to evaluate the reliability of AI-generated health information in the context of head and neck surgery.METHODS: The IT-QAMAI tool was adapted from the original English version and involved a rigorous translation and back-translation process. The validation involved 18 researchers from 13 centers across Europe, assessing 24 AI-generated responses categorized into clinical scenarios, theoretical questions, and patient inquiries. The tool’s reliability was measured using Cronbach’s alpha for internal consistency, the Intraclass Correlation Coefficient (ICC) for inter-rater reliability, and Pearson’s correlation for test-retest reliability.RESULTS: The IT-QAMAI demonstrated high internal consistency (Cronbach’s alpha = 0.850) and good inter-rater reliability (ICC=0.750). Test-retest reliability was strong (rs=0.887). Significant differences were found in the quality of AI-generated responses across different question types.CONCLUSIONS: The IT-QAMAI tool is a reliable and valid instrument for assessing the quality of AI-generated health information in Italian, with significant implications for its use in clinical practice and research in head and neck surgery.
Source: Otorhinolaryngology
Authors: Luigi A. VAIRA, Giacomo DE RIU, Giovanni SALZANO, Carlos M. CHIESA-ESTOMBA, Giuseppe CONSORTI, Giulio CIRIGNACO, Fabio MAGLITTO, Antonino MANIACI, Miguel MAYO-YANEZ, Marzia PETROCELLI, Alberto M. SAIBENE, Stefania TROISE, Andrea DE VITO, Sholem HACK, Francesco LAGANÀ, Bernardo BIANCHI, Paolo BOSCOLO-RIZZO, Jerome R. LECHIEN
Institutions: University of Sassari, University of Naples Federico II, Donostiako Unibertsitate Ospitalea, Azienda Ospedaliero-Universitaria delle Marche, Università degli Studi di Enna Kore, Complexo Hospitalario Universitario A Coruña, Ospedale Bellaria, University of Milan, University of Nicosia, Sheba Medical Center, Ospedale Policlinico San Martino, University of Trieste, University of Mons
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
OpenAlex ID: W7128030000