Work

Spatiotemporal forecasting of opioid-related fatal overdoses: towards best practices for modeling and evaluation

Year: 2024

Type: article

Abstract: To inform public health interventions, researchers have developed models to forecast opioid-related overdose mortality. These efforts often have limited overlap in the models and data sets employed, presenting challenges to assessing progress in this field. Furthermore, common error-based performance metrics, such as root mean squared error (RMSE), cannot directly assess a key modeling purpose: the identification of priority areas for interventions. We recommend a new intervention-aware performance metric, Percentage of Best Possible Reach (%BPR). We compare metrics for many published models across 2 distinct geographic settings: Cook County, Illinois, and the state of Massachusetts, assuming the budget to intervene in 100 census tracts out of 1000 in each setting. The top-performing models based on RMSE recommend areas that do not always reach the most possible overdose events. In Massachusetts, the top models preferred by %BPR could have reached 18 additional fatal overdoses per year in 2020-2021 compared to models favored by RMSE. In Cook County, the different metrics select similar top-performing models, yet other models with similar RMSE can have significant variation in %BPR. We further find that simple models often perform as well as recently published ones. We release open code and data for others to build upon.

Source: American Journal of Epidemiology

DOI: 10.1093/aje/kwae343

Authors: Kyle Heuton, Jyontika Kapoor, Shikhar Shrestha, Thomas J Stopka, Michael C Hughes

Institutions: Tufts University, Wellesley College

Language: English

Topic: Opioid Use Disorder Treatment

Subfield: Public Health, Environmental and Occupational Health

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: hybrid

Estimated APC paid: $5,099