Work

Feasibility of an artificially intelligent vaccines chatbot in Pakistan: A mixed methods evaluation

Year: 2021

Type: article

Abstract: Background Despite free vaccination services, gaps in information access impede immunization uptake in low and middle-income countries (LMICs), including Pakistan. We developed Bablibot (Babybot), a local-language, text-based chatbot to connect caregivers with immunization-related information in real-time, and evaluated its feasibility and acceptability in resolving immunization-related queries among Karachi's low-income communities. Methods We developed Bablibot using Natural Language Processing (NLP), Machine Learning, and Human in the Loop features. We conducted a sequential mixed-methods pilot in Karachi where caregivers were recruited when visiting selected immunization centers for any childhood vaccine except Measles 2 (last vaccine visit) or were informed through targeted text messages. We assessed Bablibot's feasibility and acceptability through user-engagement metrics and thematic analysis of qualitative interviews conducted with 20 pilot participants. Results Between March 9, 2020 and April 15, 2021, Bablibot accrued 677 users and participated in 874 conversations. Of 2,202 pilot-enrolled caregivers, 210 (10%) interacted with the bot; other 467 users were not pilot participants. Most queries were about due dates (25%; 221/874), delaying vaccination (17%; 146/874), or side-effect management (16%;137/874). The conversations now constitute a valuable training database for future local-language bots. Over 90% (277/307) of responses to text-based exit surveys indicated user satisfaction. Qualitative analysis showed caregivers appreciate Bablibot's convenience, reliability, and responsiveness. Conclusions Our results demonstrate the feasibility and acceptability of local-language NLP chatbots in providing immunization information. Text-based chatbots can minimize workload on helpline operators, in addition to quickly resolving queries that otherwise lead to delay or default. Bablibot's conversation data provides learning infrastructure for future health-related bots. Key messages An AI-based NLP chatbot is a feasible and acceptable intervention for providing immunization-related information to large number of caregivers 24/7 in limited resource setting. By serving as a remote, low-cost and confidential bi-directional channel, chatbots can address gaps in information access, especially for women, and have long term impact on immunization uptake.

Source: European Journal of Public Health

DOI: 10.1093/eurpub/ckab164.507