Correction: A hybrid, iterative approach, to support the development of fit-for-purpose sensor-derived measures
Year: 2026
Type: erratum
Abstract: Digital health technologies (DHTs) offer unprecedented opportunities to transform drug development. Biopharmaceutical companies are leveraging DHTs to enhance existing measures and develop novel digital measures to support drug development and increase our understanding of patient health (1)(2)(3). The passive collection of continuous, longitudinal data offers the possibility of a deeper insight into patient experience, extending beyond limited infrequent site visits, informing more nuanced trial endpoints. This approach to clinical trial endpoints also aligns with the growing emphasis on patient-centric care. By focusing on the patient's needs and experiences, healthcare providers can tailor their interventions more effectively. However, the path to approval for digital measures to support clinical trial endpoints has been slow due to siloed and fragmented efforts, limited understanding of the regulatory evidentiary requirements, and a lack of clarity on terminology and definitions (4)(5)(6).Methods that are used to measure endpoints need to be supported by data to ensure that they are producing reliable information with both internal and external validity. This is essential for approval of the endpoint by regulators and applies to DHTs in the same way as it applies to biomarkers and clinical outcome assessments (COAs). However, while the mechanisms for demonstrating this are well-defined for COA and biomarker endpoints through guidance and reviews and approvals, the newness of current guidance and inconsistent terminology around DHTs have resulted in few reviews/approvals. The era of Patient-Focused Drug Development (PFDD) presents further challenges for DHTs, as it calls for DHT endpoints to be used to support label claims exclusively when the digital measures something relevant and important to patients. This mirrors the environment for COAs, where endpoints gain traction when they align with patient priorities. In contrast, biomarkers often prioritize what can be measured, resonating more with clinicians and scientists than with patients. DHTs, while constrained in their measurement capabilities, operate in real-world environments, and provide additional insights on patients experience which is an advantage over other endpoints. The challenge lies in striking a balance: leveraging DHT data to inform endpoints while maintaining a sound measurement and clinical basis. Some advocate for a patient-led approach, while others favor a data-led approach. We believe that finding a middle ground is crucial to making DHT data appealing to regulators.To date, the instances of qualification or endorsement of DHTs by the EMA or US FDA are exceedingly rare. The most common cause for rejections of submitted letters of intent and evidence packages to the FDA is the inability to demonstrate the meaningfulness of the digital measure to patients (U.S. Food & Drug Administration, a-c). Therefore, significant gaps exist in the field. The generic methodology recommended by regulators, such as the FDA's PFDD series, for selecting and developing assessment instruments does not adequately address the unique aspects of passive monitoring DHTs. Unlike traditional COAs, which cover a wide range of subjective and objective measures, digital endpoints are constrained by the capabilities of the technology employed to capture them and the feasibility of deploying them within the context of a clinical trial. As a result, researchers must navigate a landscape where technological constraints may impede the comprehensive evaluation of treatment effectiveness and patients' experiences. In this context, following a generic methodology for all COAs may lead to inefficient processes and may not allow to take full advantage of the available technologies.An alternative sequential process for selecting and developing digital endpoints has been proposed (7). While this approach suggests that technical considerations should follow the identification of the desired measure, it may not be the most efficient in practice. Similarly, the Clinical Trials Transformation Initiative (CTTI) has introduced an "interactive selection tool" to guide the selection of DHTs. This tool emphasizes the importance of identifying relevant measures based on patient input, though it places less emphasis on technical aspects such as feasibility and usability. More recently, and in opposition to the patient-centered perspective, a data-driven approach has been proposed. This perspective is fostered by data scientists who explore the potential of devices and algorithms to capture information, revealing fresh insights and overcoming measurement limitations. In a study by Taylor and colleagues (8), researchers from F. Hoffmann-La Roche AG explored the contrasting patient-centric and data-centric strategies for converting sensor-derived data into digital clinical measures. The article delved into the nuances of these approaches, shedding light on their implications for healthcare. According to the authors, while a patient-centric outcome "uses patient insights to summarize those sensor features that are optimally relevant to patients' functioning in everyday life", a data-driven measure refers to "a digital outcome measure that uses a datadriven approach to summarize those sensor features maximally sensitive to the concept of interest". Both approaches have pros and cons. The patient-centered approach ensures relevance to patients and supports label claims when measures are valid. However, it's inefficient because it neglects technical feasibility until late in the process. On the other hand, the data-driven approach is more efficient but risks creating a digital biomarker unsuitable for endpoints. Moreover, it's methodologically vulnerable due to its departure from the scientific method and susceptibility to biases like overfitting (i.e., poor generalization to new, unseen data) and p-hacking (i.e., manipulating the data analysis to achieve statistically significant results). Recent research analyzed dozens of frameworks and identified three significant gaps in prior frameworks for assessing digital health interventions: they were not sufficiently adapted to address the unique evidence needs of digital health, lacked specific criteria for evaluating evidence quality, and rarely utilized robust methodologies developed for non-digital interventions (9). These shortcomings underscore the necessity for a more tailored and rigorous approach to effectively evaluate digital health solutions.Drawing from the limitations of these approaches and examining the differences in the development processes for conventional COAs and DHT-passive monitoring COAs, we propose to combine elements of the patient-centric and datadriven approaches to build a holistic and efficient methodology to develop digital measures that are valid, reliable, and meaningful to patients and HCPs. We call this the hybrid approach.2 Hybrid data-and-patient-centric methodology for developing novel digital measures A hybrid approach integrates elements from both patientcentric and data-centric approaches to develop and validate novel digital measures. While maintaining a patient-centered focus, and starting from what is meaningful to patients, it also acknowledges the importance of considering device and data feasibility early on.Following the COA approach and V3 framework, we recommend starting the evidence generation journey by identifying Meaningful Aspects of Health (MAHs) and Concepts of Interest (COIs) to patients through research. Afterward, appropriate measures are selected in alignment with COIs, ensuring that the relevant digital assessment instruments measure the COI that matters to patients. It is important to clarify that with the term "digital assessment instrument" (see Table 1) here we refer to the combined DHTs and derived measures that can potentially be used in a clinical trial (e.g., an actigraphy device estimating the time a day that the patient spends in performing moderate to vigorous physical activity). In theory, DHTs and derived measures are independent of each other. However, in practice, often they are closely intertwined. DHT manufacturers not only own the device but also the algorithms that transform the device's raw data into digital measures. For this reason, we define "digital assessment instruments" as the combination of the device and the derived measure. However, in scenarios where the device (i.e., data collector) and the algorithm (i.e., data transformation instrument) come from different DHTs, we can indeed envision a digital assessment instrument as a system composed of multiple instruments working together.In this methodology, the connection between MAH, COI and digital assessment instruments will inform the choice of endpoint; whether it is based on a digital biomarker, or a digital endpoint. While digital endpoints capture data related to how a patient feels, functions, or survives, biomarkers are objective and quantifiable characteristics of biological processes and do not necessarily re ect a patient's experience or overall sense of wellbeing (17). A DHT can serve as both a digital biomarker and a digital endpoint. For example, in a Phase 3 trial conducted by Bellerophon, the FDA endorsed a digital measure of the time spent in Moderate to Vigorous Physical Activity (MVPA) as a primary endpoint for subjects at risk of pulmonary hypertension associated with pulmonary fibrosis (10). This endorsement was based on the measure's clinical meaningfulness. However, if the same MVPA measure were used to predict hospitalization risk in asthmatic patients, it would be considered solely as a predictive biomarker with limited proven relevance to the patients. In the hybrid methodology, various work ows may emerge. For instance, when the association between the MAH, the COI and the digital measure is not firmly established at the early stages, the digital measure can continue its validation process with the goal of developing a biomarker.We recognize that the order in which evaluation components are considered to develop a novel digital endpoint impacts the outcome. It is crucial to begin with an unmet need rather than an unfounded urge to advance digital health or to use specific digital tools. Initiating the process by actively listening to patients, identifying the most important health-related matters, and evaluating assessment needs within the clinical context is the only way to be aligned with regulators, and the most effective methodology to guarantee patient-centric progress. Delaying assessment of feasibility aspects toward the end of the process can be highly inefficient. In many cases, this approach results in a list of measures that, while proven highly relevant to patients and clinicians, remain unattainable. For instance, results from concept elicitation activities may reveal that sleep quality is what matters most to patients with Chronic Obtrusive Pulmonary Disease, however a proper assessment using DHTs may require the collection of high-resolution EEG recordings, which in most cases would be costly, highly operationally complex, a burden to patients, and generally unfeasible. For this reason, in the hybrid approach, we advocate for a highly iterative Patient-Centric approach, a customized framework that balances patient-centricity and technological feasibility (Table 2). This approach orchestrates research activities towards two Using the Actigraph LEAP to assess physical activity using MVPA data (10) connected objectives: identifying what needs to be measured (i.e., the measure) and determining how to measure it (i.e., the digital solution).3 Demonstrating that a digital measure matters to patients and HCPsAs mentioned earlier, regulators are encouraging DHT researchers to look to the guidance on PFDD and COA as a starting point to understand evidentiary expectations for endpoints measured using DHTs. These guidances put a lot of emphasis on showing that COAs are relevant to patients. This has traditionally been done through demonstration of content validity (at least in part). Although this term is commonly used it is also misused, and the new FDA framework has not included this term, instead opting for a framework that says explicitly that the COA should assess a health concept of interest that is important to patients (component A) in a comprehensive manner (component B), and that the score from that COA should re ect the COI well (components E & F) and not be significantly affected by other factors (components C and D) (see Table 2). All of these are also relevant to DHTs and the measures derived from them, but the process may be different and there is therefore some ambiguity and uncertainty. For instance, using a smartwatch to collect total sleep time can support label claims regarding the health concept "sleep quantity". In contrast, the number of awakenings may not be enough to support claims on "sleep quality", since other aspects of sleep quality (e.g., feeling rested) would not be captured. These measures may not fully resonate with patients but can significantly aid clinical decision-making. For instance, digital biomarkers fall into this category, and provide valuable information about a patient's health status, disease progression, or response to treatment, even if patients themselves may not fully understand or appreciate their significance.Considering these limitations, we propose a selection and development methodology to collect the necessary evidence supporting the connection between MAH, COI and digital assessment instruments. This methodology employs a patientcentric iterative approach at two critical stages: first when proving the connection between the MAH, the COI, and the digital measure (stage 1) and when developing the digital measure (stage 2) when clinical validity has been established and the digital measure has become a "root endpoint". The process involves answering three key questions at each stage: (1) what to measure, (2) feasibility of measurement, (3) reliability of measurement. In detail:1. What to measure (see Figure 1)?1. Identify MAHs and derive COIs: Understanding the MAHs is a crucial starting point. Patient feedback and narratives provide valuable context and firsthand experiences related to the disease. Once this foundation is established, researchers have two primary avenues for pinpointing relevant COIs. In the first approach, stakeholders are consulted (e.g., using the Delphi Method) to narrow down specific elements of the MAH that can be practically measured. The second method involves delving into existing literature. By conducting a systematic search, researchers uncover published evidence related to patient experiences. In addition, healthcare professionals (HCPs) offer valuable insights that enhance the researchers' understanding of specific health concepts and related measures. This exploration identifies signs, symptoms, and impacts, supporting the creation of a conceptual model of the disease. The conceptual model of disease has long been a starting point for regulators to ensure that sponsors (drug developers) are measuring what matters to patients and HCPs, rather than just things that they think their product will change. Definition of the unmet need and the rational for using DHTs (e.g., Use of remote monitoring to overcome large severity uctuations in a specific symptom and improve the precision of the estimates). Feasibility studies (e.g., analysis of patterns in missing data) and complementary psychometrics supporting the construct (e.g., divergent validity).The method used to score responses in the COA should be suitable for assessing the COI accurately.Evaluation of the algorithm used to transform data into an endpoint (e.g., accuracy, precision, In some cases, it may not such as when is COA should re ect the specific health experiences related to the This ensures relevance to supporting the construct (e.g., and and data supporting meaningfulness (e.g., patients' in COA should be and in of their on patient of and to (e.g., data (e.g., may provide complementary outcome insights from patient and feedback and information from clinical with the objective of methodologies to the selected assess the of and to The evaluation of each instrument is established based on COI and regulatory other that with the term here we refer to DHTs and derived measures that can potentially be used in a clinical trial (e.g., an actigraphy device estimating the time a day that the patient spends in performing moderate to vigorous physical this involves more than just selecting digital measures. It also the and the by evaluating the connection between the MAH, the COI, and the digital measures. However, the methodology to this connection While patient and other approaches are commonly they may be in scenarios where the digital measure is and to to patients. in the are with finding the most suitable approach and will need regulatory guidance to a We propose a patientcentric iterative approach, which involves the digital measures with patients. The goal is to measure selection and digital measures that have a connection with the MAH and the However, it's important to that measures are relevant and to a healthcare These measures may on aspects that clinical (e.g., as a for developing even if they fully resonate with patients. For instance, biomarkers fall into this In such cases, we believe that the connection between the COIs, and digital measures should be with HCPs. This process that recommended for regulatory use of COA data to measure (see Figure the Once an instrument and digital measure) has been the is to define an endpoint that while maintaining A "root is a endpoint from which various endpoints can be For example, a endpoint be as the in a patient's total number of awakenings from on to on the other hand, is a specific or derived from or more endpoints. It is used to assess the effectiveness of an or the of a For instance, an endpoint be derived from the endpoint mentioned by examining the in a patient's number of awakenings from in to In this derived we two (1) the time was to align with the assessment of the trial while ensuring to and (2) the method was to the number of awakenings instead of using a the reliability and of aspects need to be considered at this For example, that when estimating measures of physical the response were when more of assessment were included and when were from the analysis Understanding how to the data analysis of a digital measure is crucial for a endpoint and other endpoints that can to the context of 2). For instance, when identifying an endpoint for measuring in using a actigraphy device in patients, considerations come into relevant and conducting a of the psychometrics different we may that become only enhance precision, data collection and within each may be on these the "root be as the of in over an at of derived measures to patients and As of the it is essential to the "root with patients or in connection to the components This involves data to demonstrate that measuring an in these specific endpoints is relevant to patients or (see in Table 2). For example, we can explore whether in activities by on to patients. that are adapted to the limitations and characteristics of the study can be derived from the "root into such as the need for data The process can different (see Figure a measure meaningful to patients, it may lead to creating a iterative process to evaluate the reliability of a selected measure and its meaningfulness. This the of a measure for use in a clinical trial as an endpoint or endpoint for label if meaningfulness established but the measure is valuable to HCPs, it a digital In cases, this biomarker may align with the same biological mechanisms as the endpoint and have potentially supporting label claims as a hybrid approach offers in digital health by patient feasibility and psychometrics researchers can advance patient-centric while maintaining measurement precision and The process with identifying what matters to patients, to guide the selection of a digital with robust psychometrics and assessing the meaningfulness of derived endpoints. a robust connection between the three of of the clinical assessment MAH, the COI, and the digital an iterative exploration process is This involves MAH and the and to digital instrument) to a will be if focusing on novel digital collection Figure alternative which we term the this This is not done at the of the but due to In the researchers a digital measure with and evaluate its meaningfulness to patients. This approach was employed by the to gain from the for deploying the endpoint in The data-driven approach is to this it can lead to A comprehensive for selecting and developing novel digital endpoints and biomarkers to be used in drug development. This methodologies for identifying what to measure and how to measure and critical if not For example, the of analysis as from data or it multiple in the same This may in the of in and the of between clinical measures. address researchers are to use and different as and to ensure independent validation of their significant in the approach is potential prior of results (e.g., digital biomarkers with clinical and about measured can in the identification of aspects and concepts by patients and other this the conducting the content validity study should remain to digital biomarker validation This approach, while is as robust as comprehensive or to patient experience or relevance as the proposed hybrid the context of the hybrid approach, iterative with regulators the development process is A well-defined process with and will allow researchers to and This approach supports the of our research to achieve of validity and digital digital or endpoints as therefore essential to expectations for different considering technological and limitations. For instance, in some cases, a connection between the MAH, COI and the digital measure may be only This when the digital measure does not fully the health concept but only a specific we should evidentiary by with regulators and in alignment with these limitations. This article the potential of a hybrid approach for developing digital measures by patient-centric with data-driven This researchers to advance patient-centric in to focusing on measurement precision and through iterative with the article emphasizes for technological limitations, and a framework to support the connection between MAH, COI, and digital measure
Source: Frontiers in Medical Technology
Institutions: IQVIA (United Kingdom), IQVIA (United States)
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,353
OpenAlex ID: W7124454588
Cites:
Cited by:
Related to: