Year: 2023
Type: conference-paper
Abstract: While dental disease is largely preventable, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. One of the main challenges in developing such an algorithm is ensuring that the algorithm considers the impact of current actions on the effectiveness of future actions (i.e., delayed effects), especially when the algorithm has been designed to run stably and autonomously in a constrained, real-world setting characterized by highly noisy, sparse data. We address this challenge by designing a quality reward that maximizes the desired health outcome (i.e., high-quality brushing) while minimizing user burden. We also highlight a procedure for optimizing the hyperparameters of the reward by building a simulation environment test bed and evaluating candidates using the test bed. The RL algorithm discussed in this paper will be deployed in Oralytics. To the best of our knowledge, Oralytics is the first mobile health study utilizing an RL algorithm designed to prevent dental disease by optimizing the delivery of motivational messages supporting oral self-care behaviors.
Authors: Anna L. Trella, Kelly W. Zhang, Inbal Nahum-Shani, Vivek Shetty, Finale Doshi-Velez, Susan A. Murphy
Language: English
Subfield: General Health Professions
Field: Health Professions
Domain: Health Sciences
Open Access status: diamond
Funders: National Science Foundation, Siebel Scholars Foundation, National Institutes of Health, National Institute of Mental Health, National Institute on Drug Abuse, National Cancer Institute, National Institute of Biomedical Imaging and Bioengineering, National Institute of Dental and Craniofacial Research, Division of Chemical, Bioengineering, Environmental, and Transport Systems
Awards: G25851218, Personalized Digital Behavior Change Interventions to Promote Oral Health, Data-driven subtyping in major depressive disorder, G4159675886, G4208625158, Novel Methods for Intensive Longitudinal Data in SMART Studies of Drug Abuse and HIV, Graduate Research Fellowship Program (GRFP), G5556446122, G5867770422, AI Institute in Dynamic Systems, Novel use of mHealth data to identify states of vulnerability and receptivity to JITAIs, G7678611020, G773633582, Personalized Digital Behavior Change Interventions to Promote Oral Health, Center for Methodologies for Adapting and Personalizing Prevention, Treatment and Recovery Services for SUD and HIV (MAPS Center), mHealth Center for Discovery, Optimization, and Translation of Temporally-Precise Interventions (mDOT), G958666079
OpenAlex ID: W4382317627