How Notifications Affect Engagement With a Behavior Change App: Results From a Micro-Randomized Trial
Year: 2023
Type: article
Study design: Clinical Trial
Abstract: BACKGROUND: Drink Less is a behavior change app to help higher-risk drinkers in the United Kingdom reduce their alcohol consumption. The app includes a daily notification asking users to "Please complete your drinks and mood diary," yet we did not understand the causal effect of the notification on engagement nor how to improve this component of Drink Less. We developed a new bank of 30 new messages to increase users' reflective motivation to engage with Drink Less. This study aimed to determine how standard and new notifications affect engagement. OBJECTIVE: Our objective was to estimate the causal effect of the notification on near-term engagement, to explore whether this effect changed over time, and to create an evidence base to further inform the optimization of the notification policy. METHODS: We conducted a micro-randomized trial (MRT) with 2 additional parallel arms. Inclusion criteria were Drink Less users who consented to participate in the trial, self-reported a baseline Alcohol Use Disorders Identification Test score of ≥8, resided in the United Kingdom, were aged ≥18 years, and reported interest in drinking less alcohol. Our MRT randomized 350 new users to test whether receiving a notification, compared with receiving no notification, increased the probability of opening the app in the subsequent hour, over the first 30 days since downloading Drink Less. Each day at 8 PM, users were randomized with a 30% probability of receiving the standard message, a 30% probability of receiving a new message, or a 40% probability of receiving no message. We additionally explored time to disengagement, with the allocation of 60% of eligible users randomized to the MRT (n=350) and 40% of eligible users randomized in equal number to the 2 parallel arms, either receiving the no notification policy (n=98) or the standard notification policy (n=121). Ancillary analyses explored effect moderation by recent states of habituation and engagement. RESULTS: Receiving a notification, compared with not receiving a notification, increased the probability of opening the app in the next hour by 3.5-fold (95% CI 2.91-4.25). Both types of messages were similarly effective. The effect of the notification did not change significantly over time. A user being in a state of already engaged lowered the new notification effect by 0.80 (95% CI 0.55-1.16), although not significantly. Across the 3 arms, time to disengagement was not significantly different. CONCLUSIONS: We found a strong near-term effect of engagement on the notification, but no overall difference in time to disengagement between users receiving the standard fixed notification, no notification at all, or the random sequence of notifications within the MRT. The strong near-term effect of the notification presents an opportunity to target notifications to increase "in-the-moment" engagement. Further optimization is required to improve the long-term engagement. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/18690.
Source: JMIR mhealth and uhealth
DOI: 10.2196/38342
Authors: Lauren Bell, Claire Garnett, Yihan Bao, Zhaoxi Cheng, Tianchen Qian, Olga Perski, Henry W W Potts, Elizabeth Williamson
Language: English
Subfield: General Health Professions
Field: Health Professions
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $2,850
Funders: Wellcome Trust, United Kingdom Clinical Research Collaboration, Cancer Research UK, National Institute for Health and Care Research, British Heart Foundation, Department of Health and Social Care, University College London, National University of Singapore, Public Health England, National Institutes of Health, Medical Research Council, Economic and Social Research Council, School for Public Health Research, National Institute on Alcohol Abuse and Alcoholism, National Institute on Drug Abuse, National Cancer Institute, National Institute of Biomedical Imaging and Bioengineering, Division of Mathematical Sciences
Awards: HOD2: Data driven semi-automated approaches to comparative effectiveness research using electronic health record data, Center of Excellence for Mobile Sensor Data-to-Knowledge (MD2K) - OVERALL, Center for Complex Data to Knowledge in Drug Abuse and HIV Behavioral Science, G2024774263, G500053460, Evaluating the effectiveness of the alcohol reduction smartphone app, Drink Less, compared with the NHS alcohol advice webpage, for the reduction of alcohol consumption among hazardous and harmful drinkers in the UK at six-month follow-up: a randomised controlled trial, G6261713276, Novel statistical methods to unlock the potential of routinely collected health data: COVID-19 & beyond, Novel use of mHealth data to identify states of vulnerability and receptivity to JITAIs, Renewal of the Network of Hubs for Trials Methodology Research., Data-Based Methods for Just-In-Time Adaptive interventions in Alcohol Use
OpenAlex ID: W4361984316
