FACEing reality: productive tensions between our epidemiological questions, methods and mission
Year: 2016
Type: article
Abstract: We appreciate the commentaries provoked by our paper.1 Our intent was to flesh out unacknowledged assumptions and problematic framings and practices pertaining to ‘causal inference’ in our field of epidemiology, and we did so with full recognition that our debates are not unique and have counterparts in virtually all disciplines involved in testing empirical claims about the world and universe in which we humans find ourselves.2–4 The commentaries indicate that we have accomplished our purpose whereby the articles, considered together, productively point to important tensions between our epidemiological questions and methods, as they relate to our mission of generating knowledge vital for improving population health and promoting health equity. In our reply, we address where we agree and disagree with each commentary. Rather than rehash arguments provided in our original paper, we elaborate only on the substantive disagreements that we consider important to the discussion under way. Our starting point, one well recognized both by practising scientists and by those who study scientific disciplines, is that use of valid methods and their development is essential for scientific advancement, even as no science, including epidemiology, is defined solely by its methods.2–4 Improved methods are essential to expand the types of data we can obtain and how we analyse them, increasing the odds for generating valid inferences. They can, as several of the commentaries stress, also help clarify which empirical questions we currently have the material capacity to ask and answer (as related to technologies of discovery and informatics), and those which we cannot - thereby motivating new work to make asking the currently unanswerable questions more tractable, as well as to provide better answers to questions already thought to be answered. We can never lose sight, however, that the questions we even think to pose, or believe we can ask, depend first and foremost on the theories, concepts and material evidence that inform each and every scientific effort to explain the material workings of our world.4–6 In our field, involving dynamic populations of people in dynamic societies and ecosystems, methodical triangulation of diverse types of evidence from diverse types of study designs and involving diverse populations is essential,7,8 as is recognition that any ‘causal effect’ detected in a given empirical study is the effect estimated in that study - it is not ‘the’ (as in: ‘universal’) ‘causal effect’. Indeed, as Sander Greenland and Jamie Robins observed back in 1988 (p. 1195)9: … the dependency of epidemiologic measures on cofactor distributions points out the need to avoid considering such measures as biological constants. Rather, epidemiologic measures are characteristics of particular populations under particular conditions … Or, as the Professor of Social Biology and statistician, experimental embryologist and evolutionary biologist Lancelot Hogben stated in the conclusion to his 1933 opus Nature and Nurture, which robustly challenged the era’s dominant eugenics orientation to the analysis of population health (p. 121):11, The application of statistical technique in the study of human inheritance is beset with pitfalls. On the one hand the experimental difficulties of the subject-matter necessitate recourse to mathematical refinements which can be dispensed with in animal breeding. On the other there is the danger of concealing assumptions which have no factual basis behind an impressive façade of flawless algebra. The student may well recall the words of Wilhelm Ostwald: ‘Among scientific articles there are to be found not a few wherein the logic and mathematics are faultless but which for all that are worthless, because the assumptions and hypotheses upon which the faultless logic and mathematics rest do not correspond to actuality’. We agree: that DAGs (directed acyclic graphs) can be very useful tools, that tools are not theories–and that critical deep substantive knowledge is required to use these tools well. As Blakely et al. note, use of DAGs can be ‘meaningless and perhaps even misleading without a well-formed theory of why one node causes another’.12 We also agree, as per their Figure 1, that science necessarily involves epistemology–defined by the Oxford English Dictionary as: ‘the theory of knowledge and understanding, esp. with regards to its methods, validity, and scope, and the distinction between justified belief and opinion’.17 Equally necessary, however, is engaging with ontology: ‘the science or study of being’,17 referring to the understandings of the phenomena and processes of our world, including their causal relationships and the apt contrasts that can make them visible.4,5,10 If our paper succeeds in cautioning against, in Blakely et al.’s words, a ‘blinkered approach’ to using DAGs while clarifying where and how DAGs can be used well in ‘situations where it is apposite to apply’,12 we are satisfied. We appreciate five key points raised by VanderWeele’s thoughtful commentary. These pertain to the commentary’s: distinction between what may be theorized to be a ‘cause’ and what may, more narrowly, comprise a ‘causal effect estimand’ (point #1); recognition that epidemiology tackles many problems that involve ‘composite exposures’, thereby making it ‘difficult to speak of the causal effect estimand’, a problem further compounded by it being difficult to uphold the stringent assumption of ‘no unmeasured confounding’ (point #3), let alone no misclassification or uncertainty in measurement! –which are critical issues we address further on, including in Textboxes 1, 2 and 3; acknowledgement of the value, in the case of composite exposures, of considering the impact of multiple types of interventions and also the value of recognizing when it is appropriate to ‘abandon any attempt at a precise interpretation of a quantitative estimand and simply speak of evidence for general causation, for something to be a cause’ (point #3); realization that there are ‘important questions that are not to the as health of such as more to and even the is more apt for and (point and that that is than causal that to the is important in causal and diverse types of evidence can and be and that there can be scientists as to what the a that cannot be solely by use of such can be (point The which we is that it is essential to the of any particular of methods and the questions that can be using them, without as questions that cannot be using these We however, with key of how the scientific and in point the science as being an in which as of and data on these which ‘the may or causal a which that the use for to ‘the between these of and the In point it for causal analysis the and of the study of in the and are never simply a that are simply and as we and other causal in to critical substantive knowledge is required for the and hypotheses in the first not to of study and let alone and the of not with even as or are on an of the of and not only are there debates what are of and they can be as the of but also are not to engaging with and consider only the of such and phenomena such as and of the of dynamic engaging with other both in the and in while their as they and In where dynamic and phenomena comprise the of such as epidemiology or few be to precise ‘causal that are to the in which people and other there are no human it is one of its of and no humans they at of than for of their of what epidemiology however, the more of and that are a or on the under and our methods is that (point a it the the causal effect for the a is at odds with the of that it is a population We that the of causal is for the of epidemiological the of an causal effect is simply there are that to the in all as per the provided a or as well as more the of with for those a of as with who do not these of can be as not from the distinction between and causal a distinction that epidemiological claims for the it to what was (point and … in out of of the or causal (point the assumptions in the application of these of The of ‘no unmeasured confounding’ (point not only even for those that have but also is an that is in any epidemiological of (as to the of a on causal for using biological data is what epidemiological data on and observed that the assumption that are without is a assumption that is required causal analysis not by who these about the and assumptions of methods is to as well as to issues more the in we think the problem is the of as a by those who that in a be and problem can be by that is a in the that be by or one or that also such as (point As we and as the ‘the to not We not the why it is to as an let alone to a of only of the in that we the provided of our original of that cannot be a ‘cause’ in a because it is not a that necessarily and is an a Our point, the of epidemiological with how including by the of as an biological which to to we that we are that using to how as by the study in which one of Indeed, clarifying how was the in the first of paper we a causal to of as by and by an that not per are the of health that and in (p. If scientific as an of only to and be to and that scientific need only a methods to the of its causal that is his We with to that we are a a our to Robins and and also Textboxes a on the value of scientific and we Robins and with the issues we and other scientists in the difficult work of and testing causal they think the are is by the of their we to that many of the of not be that is to be is to is of our in by to the in which that only a not solely as the of to the and the rest of the and it in the of the a the of the Social the and a we are as to in the Robins and as the and as the for that the or the of the in as his we are to let those particular we are to of the that causal methods more than they of problems their of those in the of and are involve very can be by of the interventions issues of health to of a to only problems is not a as Robins and on or the of better methods by not the of the that can help to on the of making problems more tractable, at to the point of being to to general about per VanderWeele’s even generating a precise ‘causal estimand’ is not the of is in and who as of the in the and also to the and of the of and are (p. In the the a from the to the when a of people but thought that to be more what they thought that the to be and with The of was an at the in of a of than and of them on a of five or the of a is not at all that the important questions in and to necessarily on a of than a or a of or We find it that Robins and to the as the to that it As we it is in a as we one in the epidemiological their of the of the used in to is In the case of empirical in which evidence is being the are to what to be not the it is that the theorized causal in the world we is a from that of the of because of how and for for a study of in The of that Robins and several issues we have with their in the commentaries by and et the methods of causal that are not only by other can provide or of causal that are by other The of and as we explain in 1, our that in of causal processes that for and health depend more on in the material basis of human than in how we of such we also find it that Robins and find our of empirical to be of the a the need for deep substantive knowledge about the theorized causal also new scientific to as a of how we our claims about ‘causal because we cannot all causal relationships and have we the Robins and that is to do only that such an evidence in and The point is that data are never simply of (as per VanderWeele’s application of methods to including causal methods, can only to generating knowledge there is about the of the involved and their causal DAGs have for of and the and of from and to the about because of deep biological knowledge with causal but not to one to be the consider only the of and including those using which is an and which never consider asking how health and not only the but who is in them, and by make a the theory and deep substantive knowledge of the of the phenomena under study are as is knowledge about the their we Robins and pertaining to such questions as: our we and what is the impact of on the of triangulation of evidence from diverse that we and that any of health impact of and have is the world we the world in which we are to population health and health equity. As we not only have to be about uncertainty and the of our but also be about to the evidence we can, for that more even of the provided by the the we have to the assumptions with that dominant that We make no that and necessarily point in the of health the a and of types of These tensions between our methods and mission are not and do we new about their and we do is a that these tensions a that to be in each and every and we are to our Or, as in his 1933 and (p. is is more and with a for the of human societies and of the who to a basis for the of our a that an of the material and of the issues we analyse in epidemiology may be by an on the methods of how we our We in the by and in the of in and about the between biological and statistical that analysis in is being for scientific can its effect’ and also that to to believe that the the from an and no other the in that work in epidemiology a for methods, and the of that do to a and that at to all that is and in epidemiology under an of mathematical and a We these as they the that was to in the epidemiological Our analysis of from to that science not need theory at by et that science not only pertaining to methods but also pertaining to as well as an to from only not to however, we are to that et al. do not that interventions be currently or that the is ‘the of study for causal of the first is their that the causal of can be a particular not have even such a is to we find et al.’s discussion about health involving and to be and problem is compounded by their of raised as of a in a a with the on health let alone the they on to the in to their stated that a to quantitative in necessarily ‘the by subject-matter to in et al. solely in and as of to and that the is a by in the and on the that in the of the of and that the other of in from original of a not only in the of evidence the of but also how a scientific on the assumption that is an a biological of also from who to to as being or both and the have a and who by as to which and to and can by by by by the in which they an that the of the of its to and is to that the of considering an is raised by a of the or the point of as a of population what people in a on than one which in which The that what is ‘the of is not a for the impact of the both and the and societies in which they are and in conditions and with in and with and by and These are the which are and which they is a of what we to that we that can and be as a ‘cause’ the that it is not is In we the Our is that not be considered a because of the arguments about it being but is not a valid biological and the and practices that with its are the causal phenomena of is how in its many can be and are in the on is an on and as of health and health which the as to who and what may and health knowledge to be both and an of how they be is to be Our about the of DAGs in causal are we think DAGs can be and (as et al. to that substantive knowledge and causal are required for a to be than DAGs not the for because even the is there be an to what can be to a as when have the every problem can a of to all at causal from to and other We further that in our to what et al. of the DAGs in the or in the do not to the deep being in all the problem of whereby causal DAGs can be with or the of that and all and unmeasured while being In Textboxes 2 and we - pertaining to and and to and many that have which the use of DAGs not to these and which also make no attempt at triangulation or to the As of with DAGs is that the causal is not and is a problem because on the of the causal how the (as by the its are so be its causal If our paper more critical thought as to what DAGs is so the Indeed, the DAGs that et al. in their are of the problems at for the their Figure the point that the DAGs that have never have all the their one cannot do science and knowledge on the In the the of DAGs that have DAGs back was only to in with his work on analysis in the including his recognition of the need to that of of and have and have the key that was in the causal as by the of and of with and the we one and perhaps to a is to have the answer in The analysis of the for the is The by that et al. and which we in of the we was only DAGs to that the be as to with As we in our paper, the value of VanderWeele’s is that it that DAGs causal the is a that DAGs help that the is in effect a whereby use of the DAGs was stated to that can be as to on a that is by the of and those causes with the the of the by et however, DAGs have to an all which is not the the of a problem is not a to better Our paper and the commentaries do a the need for thoughtful science that can address the questions of our also to to why we may have our paper as we at on the for the for relationships between Blakely et al. that epidemiology it be to solely on DAGs and to an and of scope, but more of how the health of a population is observed that the of the may have in questions are not to a perhaps a problem which in it is important to and in the more of questions and are Robins and the theory and application of causal methods is and it is that many such are being such that on these to all recognition that many important problems with for population health cannot be by these methods, and the problem be As other scientific and which these we can our field to the of the we can that a to discovery and of of population health and health to expand the and on In the Professor of and at the of and his and to the of not only of the dominant of and epidemiology, but also upon involving and the however, was more because in words, ‘the (p. As there was not why of to general epidemiological … but … such was the effect of the (p. The is as we the of epidemiological knowledge application of causal and methods, both of which are the of deep scientific and we what we from our knowledge of our and from our study of the and of it be to epidemiology solely to use of and the questions it can answer with was in by Professor the at the of The of and what to causal from the as and that for the a general in of to the of of to the of many of of In these one of a paper which that it was not using to of any of and given issues involving analysis of exposures, in particular of which for to and in an effort that the effect was to the effect was The that the that at as well as may an as to what of paper by of the causal field used and as a and at but the of as a the only a few these on data challenged by new was not new of the problem the was the of an on in which of ‘the of from to that and as was as a more to causal in the of by and with with not as the the and data per not was in with an of of several to is not a unique several other have in which epidemiological and other data there be a effect of a but and in their the of more of of of are including in which have the testing the causal and it is that the be in an when such can be to the claims that of the or causal are key for in epidemiological by other improving from in material understandings of how the world and of the of to the material world, not The by that Robins and perhaps have in is not is to be but and where the of a the is ‘causal the case of and study in in the of the and the help of a analysis of which the effect of on The analysis the as a knowledge of the field the of as a to be a a the a of evidence that is not causal with to that have that is a causal have by the of problems that are to valid causal in the and effect of case an and with and from a of however, no discussion of these problems in the paper at per the quantitative of ‘the to quantitative in the that for of the effect of on is a quantitative causal is cannot be a to causal problem was by the of use of triangulation or to the to the by several of the on our that both triangulation and are the that already these a on the paper its use of DAGs and with the causal effect of on health is it an to have precise but As with the in 1, other to analysis that use causal than statistical can more evidence on as by an analysis of the as in the paper under and are not a for subject-matter In a promoting the use of causal in epidemiology, a the causal for the that no effect on as the where we have measures of but do not have measures of when the or the for the we for the for a is a to these in and The conclusion is that for a statistical between and as be on a and the one not as a an is that be in and other that development and of of for which there is a of be and in not for as be a for the between of the and the of (as both and not the by subject-matter that et al. out in and the of DAGs that be for a of analysis be which out an analysis for every for the DAGs that thought for the DAGs measures for which have no We that these are issues for epidemiology, and that the that be in the of analysis is a one such are on knowledge and not be for the of the what in We consider it that the use of DAGs not ‘the by subject-matter is the that DAGs currently as a to triangulation how we the DAGs are that the data the have to hand not data they cannot that are by and unmeasured that do not their any attempt at triangulation or to the and that the of as the of epidemiology for As one case in point, a analysis of and was and the as be from such that the health be with the of and discussion of the DAGs very Our in to et al.’s that are by what and to the is by how and useful for such as are The an to and many a of the used the to find a We to these and many other such can be the that to their as to the application of subject-matter knowledge and triangulation to causal and et al.’s in of the of be by data with characteristics of the that are and the that to such of one of these that is a between and as with the that between and in with no acknowledgement of the evidence not also with to analysis all the problems in and the of from the data a with an to the of and for which all between it and the are by also that only these when on a of The use of that are than from is to to misleading given the impact of characteristics on such The that such of not it The of these in our the of causal in paper to the of who to think about science, to do science to is as an for and to think about the health in to and of
DOI: 10.1093/ije/dyw330
Authors: Nancy Krieger, George Davey Smith
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W2579903534