Year: 2021
Type: article
Abstract: Electronic health records (EHRs) are now ubiquitous in US medical care, in part owing to legislation that incentivized their adoption as a component of the 2009 American Recovery and Reinvestment Act. 1 These records have been associated with improved medical communication through increased legibility and rapid access to individual patient information.However, the promise of leveraging the rich digital data collected during routine clinical practice for research and to support point-of-care applications is only recently being achieved.This progress is enabled by technology that augments traditional abstraction, software that digitizes and interprets unstructured documents, and computational tools that rapidly analyze large data sets.The report by Yuan and colleagues 2 highlights the promise and challenges of applying artificial intelligence tools in the interpretation of real-world data derived from EHRs.This research addresses 2 important goals: construction of realworld data cohorts and predictive modeling.In oncology, a small minority of patients take part in prospective clinical trials of investigational therapies, and those who do tend to be younger, have fewer comorbidities, and be less sociodemographically diverse than the broad population of patients with cancer.Thus, the generalizability of clinical trial results may be questioned.It may be possible to use real-world data from EHRs to explore treatment patterns and associated outcomes among a more representative population.However, many clinical features needed to select a cohort or perform an analysis, including cancer diagnosis, stage, biomarker profile, functional status, treatment, and clinical outcomes, may not be available in structured EHR data with high accuracy or completeness.This has created a need to develop technology-driven approaches for extracting information from unstructured EHR data, such as clinic notes, diagnostic test reports, and other text.Yuan et al 2 demonstrate such an approach using natural language processing in the construction of a cohort of patients with lung cancer.The authors used a semisupervised machine learning algorithm called PheCAP that supplements a small set of criterion standard labeled data with a large amount of unlabeled data to identify eligible patients.Yuan et al next extracted additional variables that were not reliably available from structured data.They used rule-based methods for many variables, such as stage, and they used machine learning for smoking status, for which criterion standard labeled data for model training was available.To evaluate their cohort selection and variable extraction, the authors used variable-specific validation measures, including sensitivity, specificity, and completeness, and added a holistic test of performance by comparing results with data that were previously collected for another epidemiologic study.Overall, Yuan and colleagues found that their constructed cohort, while limited by a sensitivity of 75% for identifying patients with lung cancer at the target specificity of 90%, achieved higher data accuracy and completeness than use of structured data alone.This work suggests that it may be possible to construct broad cohorts with a machine learning algorithm and that care must be taken to characterize the performance of the cohort.Given that such cohorts may be the basis for research or quality-improvement initiatives that ultimately affect patients, a question arises about what level of model performance is good enough.An algorithm that identifies patients with 90% specificity may be suitable for certain research questions but not others.When a very rare cohort is being identified for retrospective research or when patients who may be eligible for a prospective clinical trial are identified in real time, it may be more important to +
Source: JAMA Network Open
Authors: Neal J. Meropol, Janet Donegan, Alexander S. Rich
Institution: Flatiron Health (United States)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,000
OpenAlex ID: W3180009747