Year: 2024
Type: article
Abstract: We wish to applaud the insightful commentary provided by Didelez1 on our comparative study of data-driven versus theory-driven approaches for constructing causal life-course models.2 Didelez’ concise and honest description of both possibilities and limitations of causal discovery for epidemiology will be a tremendously useful resource moving forward. We will address a question posed by Didelez regarding the expert consensus meeting and provide a few additional topics for further research that we believe will aid both epidemiology and causal discovery research. One of the authors attended the discussions between experts as an observer. Although the experts agreed on many proposed direct causal effects, they shared frustrations concerning missing variables, including family history and adversities in childhood and adolescence. They were also frustrated about not being allowed to draw bidirectional arrows, which gave rise to discussions about which direction certain arrows should take. Moreover, in some cases, they disagreed on the degree of evidence for a given association and its causal direction. This underlines the difficulty of directed acyclic graph (DAG) construction, as well as the artificial setting in which our expert DAGs were constructed; the main focus when designing our study was to facilitate straightforward comparison between data-driven and theory-driven approaches. It would be interesting to conduct a more “natural” study in which experts first construct DAGs as they usually would and then try to identify data for a causal discovery algorithm to solve the same problem. Some work has been dedicated to structuring the DAG generation process, providing guidelines for using existing literature3 or collectives of domain experts including researchers, clinicians, patients, and others.4 A similar line of research has developed in dynamic systems modeling where causal loop diagrams have been constructed by use of the group model building framework (eg, see Uleman et al5). We find such work valuable. However, we still have little knowledge about what factors affect DAG generation and, hence, how it may be optimized. Ironically, we do not need advanced machine learning methods to address this. A few traditional randomized studies will do! We may assign different DAG construction “settings” randomly to different participating domain experts, for example, varying instructions, variables, tools, numbers of experts, software, and access to causal discovery procedures and empirical data, and thereby gain insights into causal effects of these factors. Considering how time consuming DAG generation is, we believe this meta-methodologic research would be worthwhile. For methodological causal discovery research, knowing more about the traditional approach (ie, a realistic benchmark) would undoubtedly also be useful. Causal discovery research has been immensely active, especially in the past decade, producing a large number of algorithms that show promising results when evaluated on synthetic (simulated) data.6,-11 However, this young research field is already experiencing symptoms of a reproducibility crisis: top-performing algorithms fare worse than expected on new data,12 and synthetic evaluations reflect the real world poorly: data simulations often make assumptions that researchers do not realize, and the evaluations of the algorithms can be quite sensitive to those assumptions.13,14 A more meaningful benchmarking question is whether causal discovery is a useful aid for actual scientific progress, gaining new insights, and guiding conversations among, for example, epidemiologists. The only way to answer this question is to apply causal discovery in practice—again and again—and report the results transparently and critically. We hope our study,2 Didelez’ commentary,1 and the references therein can help make this possible and, most importantly, that applying these causal discovery algorithms will ultimately teach us something new about causes of health and disease. This work was funded by Independent Research Fund Denmark grant 8020-00031B and National Institutes of Health contract R01HL159805. None declared.
Source: American Journal of Epidemiology
DOI: 10.1093/aje/kwae101
Authors: Anne Helby Petersen, Claus Thorn Ekstrøm, Peter Spirtes, Merete Osler
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4399304175