Year: 2002
Type: article
Abstract: In a commentary written several years ago [1], Professor D. Rubin and I pointed out that one is usually advised to analyse clinical trials – the preferred modern strategy for empirical evaluation of medical therapies [2, 3]– for significance levels and estimates using only, or at least primarily, the intention-to-treat principle (see, e.g. [4]). We noted, however, that the intention-to-treat estimator (see below) estimates so-called 'use-effectiveness', the causal effect on outcome of prescribing the drug, rather than the medically more important 'method-effectiveness', the causal effect on outcome of actually taking the drug. We argued that studies should be designed so that they can yield valid estimates of method-effectiveness, although so long as protocols are not followed exactly, such estimates may depend heavily on additional assumptions (scientific models – see section on longitudinal data with dropout, below); ones that go beyond the data at hand. In this paper, I present a conceptual framework for thinking about causal estimands and estimators for both use and method effectiveness, with emphasis on the latter, focusing on the problems posed by deviations from protocol, notably noncompliance and dropout. The paper is a review and exposition, rather than a presentation of original material. Section 2 presents a conceptual framework, setting context, defining causal estimands (population quantities describing causal effects of treatments) for use effectiveness and method effectiveness, how these change in the face of noncompliance, and the problem created by dropout; section 3 discusses estimators of the causal estimands defined in section 2; section 4 considers the effect of dropout on the estimators; and section 5 discusses implications for study design. Although different in some details, the ideas, point of view, and notation presented here are essentially those presented in the statistical literature by D. Rubin and colleagues, based on the Rubin Causal Model. Rather than provide extensive references throughout this work, I limit further citations to important additional works which discuss specific points or present examples, and offer here a minimal set of references which provide background and an entree to the relevant literature [5, 6], and a more complete and technical discussion of noncompliance [1, 7-9], dropout [10, 11], and scientific modelling of drug dose-concentration-response [12, 13]. The context is a clinical trial in which n individuals are nominally assigned either to a control treatment, coded 0, or a test treatment, coded 1, and a univariate outcome is observed. The control and test treatments are assumed to differ. The random variable indicating nominal assignment for individual i is Zi, and the random variable denoting the individual's outcome is Yi. A key design feature of randomized clinical trials is 'random assignment' to nominal treatment. Since the validity of every estimator discussed in section 3 depends on this assumption, it deserves to be stated formally: it is assumed henceforth that the probability of being assigned to test (control) treatment is independent of pretrial prognosis and all other baseline patient characteristics. Two types of deviation from protocol will be of concern to this paper, noncompliance and dropout, and I introduce next the notation required to deal with them. Non-compliance recognizes that nominal assignment, the treatment assigned by the experimenter, may differ from actual assignment, the treatment taken by the patient. The random variable indicating actual assignment is Di; it can be written Di(Zi) to explicitly recognize that it is a function of assignment. Di can take the value 0, 1 or 'other', where the first two values correspond to values of Z– control (0) and test (1) – and the last indicates no particular treatment, but simply that actual treatment is unknown; some alternative is pursued. Thus, while recognizing only one 'version' of control and test treatments, I acknowledge the possibility of multiple versions of the alternative. In general, Y will be a function of D (and hence Z). Let Yi(d) denote the potential outcome (see below) for individual i actually receiving treatment Di=d. To maintain parallelism with dropout (below), I introduce the (redundant) random variable ), where I(·) is the indicator function, taking the value 1 if its logical argument is true, and 0 if it is not, to designate whether the intended outcome is observed (Ui=1) or not (Ui=0). I assume that the compliance tendency of each individual in the circumstances of the trial is a (unknown) baseline variable, Vi, which determines the values of Ui(z) according to Table 1. Dropout, as a problem for analysis and interpretation of a clinical trial herein refers to a data set that is incomplete due to the loss of the outcome datum from one or more subjects. The random variable indicating responder status is Ri. It takes the value 1 if the outcome of individual i is observed, and 0 if it is not. Just as with Di, Ri can be a function of assignment: Ri(z) is the potentially observable variable indicating whether or not individual i will drop out on assignment to treatment z. Similarly to Vi, the responder tendency of each individual in the circumstances of the trial is a (unknown) baseline variable, Wi, which determines the values of Ri(z), as in Table 2. For each individual in the trial, there are 11 potentially observable quantities, Zi, Vi, Wi, and Di(z), Ui(z), Ri(z), and Yi(z), for z=0,1, although there are only 7 independent choices, as Di(z) fully determines both Ui(z) and Vi, and Ri(z) fully determines Wi. I refer to the N×11 matrix of potentially observable data as the potential data matrix, or just the potential data. In any actual trial, we observe, for example, only one of Yi(Di(0)) or Yi(Di(1)) (these might or might not be different see Table 1), depending on whether zi is 0 or 1. The Rubin Causal Model claims that even if the control treatment was assigned and we observed Yi(Di(0)), nonetheless the test treatment could have been assigned, and in that case we would have observed Yi(Di(1)). Counterfactuals, that is unobserved potential outcomes, such as outcomes on unassigned treatments, are central in the Rubin causal model, as the model defines causal estimands (population quantities describing causal effects) using them. Although the idea of potential outcomes seems natural and intuitive to most persons, caution must be taken when they are used as a basis for causal reasoning. The only kinds of potential outcomes that may meaningfully feature in the definition of the causal relationships that a clinical trial might establish are those that would result from decisions made after study inception and before study termination11 A strict view would limit potential outcomes to only those outcomes consequent on actions assigned randomly by the experimenter, but I will not so limit the discussion herein. . Clearly, the 'decision' to assign test or control treatment to an individual could be altered after enrolment, so that the potential outcome on the control treatment for a type a complier assigned to the test treatment is certainly a meaningful concept. A bit less certainly so are the potential outcomes on both treatments for a type n complier: if through added inducements and reminders incorporated into a modified study design, one could imagine altering his 'decision' not to comply with any assigned treatment, then they may be meaningful, but if not, not. However, there is no possibility of altering a possible pre-existing correlation, perhaps a genetic-cultural link, between a propensity to behave as a type n complier and more rapid drug metabolism, leading to reduced sensitivity to the test treatment if it were taken. Thus, the type n complier's potential outcome on the test treatment if he were a type a complier is not a meaningful potential outcome, and hence causality in the sense of the Rubin causal model cannot be associated with complier type. Of the 11 potentially observable variables for any one individual, we always observe three, zi=Zi, di =Di(zi), and ri =Ri(zi). If ri = 1, then we also observe yi= Yi(di). The N×3 or 4 matrix of observed data is called the actual data matrix or the actual data. Note in particular that, contrary to usual practice, di is assumed to be observed: modern methods of monitoring compliance make this assumption realistic [14]. The characteristics of such methods, in fact, determine the definition of 'actual assignment' I have used in the face of noncompliance (and in this way, this work differs from that of the other authors cited previously): modern compliance monitoring methods record medicine container openings. They can therefore indicate that an individual is not taking the prescribed medication as instructed, but if they do so indicate, they do not indicate what other treatment he is taking. Thus, as noted above, the actual assignment category –'other' denotes some version of alternative treatment, but does not specify what it is. This view is particularly applicable, for example, when Di(z)=z is defined as >80% of prescribed doses taken correctly, as is commonly done. In that case an individual who takes less than 80% (but more than 0%) of the test treatment when Zi=1 and less than 80% (but more than 0%) of the control treatment when Zi=0 is of compliance type n, and for him by definition both Di(1)= 'other' and Di(0)= 'other'; yet the two treatments are clearly not the same, nor, in general, would they be expected to lead to the same outcome. Both noncompliance and dropout can be viewed as particular instances of missing data; data that were scheduled to be observed but were not in fact observed. Here, those data are Yi(zi) for those i designated as having missing outcomes by an observation indicator as follows. If noncompliance, but not dropout is present, i such that are missing; if dropout, but not noncompliance is present, i such that are missing; and if both are present, i such that ri ui11 A strict view would limit potential outcomes to only those outcomes consequent on actions assigned randomly by the experimenter, but I will not so limit the discussion herein. ≠1 are missing. Missing data may be classified by the relationship between the observation indicator and the rest of the potential data. Depending on the class, missing data present minor (loss of precision) or major (validity) problems. Data are Missing Completely At Random (MCAR) if the missingness is independent of all other data. Data are Missing At Random (MAR) if the missingness depends only on other observed data and not at all on unobserved or potential data, including the missing data themselves. So-called nonignorable22 This term has a technical origin. Strictly speaking ignorable missing data is a fourth type of missing data. Such data are MAR, but also the Distinct Parameters (DP) assumption holds: the missingness mechanism (probability distribution of the observation indicator) shares no parameters with the data mechanism. The term 'ignorable' stems from the fact that with probability model-based estimators and ignorable missingness, the model for the missingness mechanism need not appear in the full probability model for the estimator to be valid and efficient; the missingness can thus be 'ignored'. We will not consider this detail further: MAR alone is sufficient to justify estimator validity; the DP assumption affects only precision. missing data are neither MAR nor MCAR, and this is the class that presents serious problems. Note that it may be the case that missingness depends unconditionally on unobserved data, but not on such data after conditioning on the observed data. In that case the missingness is MAR, not MCAR. In such a case, of course, for the method of analysis of the MAR data to be valid, it must correctly condition on the observed data. Thus, although MAR data can be validly analysed, such analyses are more sensitive to modelling assumptions than are analyses based on complete data or data that are only MCAR. Both noncompliance and dropout can separately cause nonignorable missingness, and this is why they present a problem for clinical trial interpretation. A simple example makes the problem with nonignorable missingness obvious: Consider a randomized trial of a new drug vs placebo for a chronic disease, with outcome being a measure of disease severity after a fixed time on treatment. If patients drop out (fail to comply) on the test treatment because of toxic side-effects, and if greater toxicity is associated with greater efficacy, then the best potential outcomes on the new drug in the group assigned to take it will be missing from the actual outcomes in that group due to the dropout (noncompliance). Therefore a natural estimator of drug effect, the difference in average disease severity between treated and controls among those providing outcome data (among those on the assigned treatment) will be downwardly biased relative to the value it would have had if dropout (noncompliance) had not been present. The missing severity measures (outcomes) are nonignorable because their probability of being missed is related to their value (through their association with toxicity) within treatment groups; that is, after controlling for the observed data (assignment). This discussion focuses on population-level causal estimands, that is, expected values of population characteristics. However, for simplicity, and because it emphasizes ideas rather than technique, I assume henceforth that the trial is large enough so that I can focus on valid estimation without worrying about small-sample variability. Given this assumption, trial-level causal estimands defined on averages over all N rows of the potential data matrix are valid (unbiased) estimators of the corresponding population estimands, and for this reason, in an abuse of notation, no distinction is made between the two. Before proceeding further, I define some functions that will prove useful. Let A and B be the vectors (a1, a2, …, ai, …, aN) and (b1, b2, …, bi, …, bN), respectively. The elements of the former are logical, in which case the the is in which case the assigned the denote the of the study population for which is and denote the average outcome with actual assignment over those individuals in the study population for is For I define the logical condition A that all study in the to be the by the where it will cause no as an argument in the above, in an abuse of notation, I denote A and denote so that for example, . is the expected value of the difference between outcome with nominal assignment to the test treatment and with nominal assignment to It use effectiveness in the population of the one from which the trial are method effectiveness is less an of clinical trials than use effectiveness, and because the of this paper is to its and discuss how it can be validly I take the here to discuss at greater why it is an important causal effectiveness – the expected outcome difference due to treatment with a new drug vs a alternative – is certainly a more important population for of individual patients than is use effectiveness – the expected outcome difference due to prescribing the new drug vs the – for a of effectiveness is a rather than a of and one can more its from trial to patients – the for on clinical trials before the use of a drug – than that and is more on specific to a particular that may not a method effectiveness both and for example, if a trial the data do not a that use effectiveness is greater than the data may be with method effectiveness, indicating that the trial may have because of noncompliance or dropout rather than because of of method effectiveness for a specific that is with in values for that a to interpretation. for drug The of method effectiveness is sufficient to the for drug is neither a nor a to use for A relationship the expected and toxicity of a drug over of compliance in a particular It is to see how this can be used to a for a new patient. In an argument can be made that a for a new patient is one that is to yield without toxicity if taken as A relationship the basis for such a emphasis on studies so that analyses will be valid, the to design that valid estimates of method I to this point to method effectiveness estimands, first that if all individuals were the method effectiveness would be however, the of actual treatments taken in to the two nominal can be different for all 4 types of 4 method effectiveness estimands may be one for each compliance Note that does not to the 4 types of does not actual treatment, so that this need only concern with the method effectiveness estimands by Of least are and the former is because it does not outcomes on either actual treatment for the is of only if the control treatment is of and if the alternative treatment by individuals when assigned to the test treatment in fact differs from the control treatment. The two method effectiveness estimands and and are of the assumption that the alternative treatment taken by type when assigned to control is the test treatment to be true, for example, when to the test treatment is to those assigned to as is in new drug – see both estimands expected outcomes the test treatment to some alternative. is of greater than as it the test treatment to the control treatment, which was for the trial for a the most relevant method effectiveness is a average of and called the Causal It is the expected difference in outcome associated with a difference in nominal assignment in that the of the trial, will take the test treatment when assigned to as above, take the test treatment when they are assigned to the control then the that is of all those individuals who will take the test treatment trial if and only if they are assigned to I and as the method effectiveness estimands of are defined on potential data, estimators are to actual data. For simplicity, I the assumption for the potential data made in the section to to of outcomes observed in each assignment group the actual data. discussion henceforth can be to estimators that are for their For simplicity, I consider first valid estimators of and in the of missing data, and then the same estimators in its Just as for randomized assignment that were the of its e.g. and ), would be valid estimators of the corresponding causal estimands, and hence would be validly by estimators into The however, is that is not The of this considers how and might validly be . Note that the assumption neither nor it does for both and will in the type of clinical trial discussed by that the 1. with test treatment is all or 2. who does not take the test treatment takes the control The test treatment is not to are to be in corresponding to each all of the (1) is for example a or a The control treatment is a defined of including individuals do for the condition defining study and The test treatment is to and cannot be without provide an The effect of is to make all alternative treatments the same and to the control treatment, and hence to make complier types and n a and even on potential only two types of types a and the always take and the former always take control assigned to Since the so defined always take the same treatment of assignment, it is to imagine how their outcome could depend on that assignment. If it does not, then the assumption the the assumption is as the This from the in which the estimator was first It is so because the assumption an effect of compliance type on outcome as it determines the treatment actually . Note also that when the assumption there is no need to observe so long as a valid of is perhaps from some other of test treatment in the group assigned to test types a and If these same complier types could also be in the group assigned to the control treatment, to just such would validly Of course, the key data – compliance unassigned treatments – are so that there is no to a and from n and in the group into a and vs n and but a may be that is observable in all if not an indicator of compliance type at least a vs is at least with that is commonly used in this is the fact that it does not do the without further provide an example of such an assumption and an associated for the control to with the among the the clinical trial data cannot provide that the assumption is If is used additional to to to this defines the to be discussed in the next in this is for method effectiveness if one can assume that the average outcome on control is the same for and type so the which that control could be to take the control treatment. Of course, they but then and without any additional the estimator can be a to use di when are not is to to if the two are it can be argued that it is that compliance and would be the same by perhaps neither group If not, the estimator can be but more a large between the two might be used to out use of the possibility is to all to the test treatment for a pretrial and to use the observed compliance in that as the course, it would be more to use the pretrial compliance as an for the rather than as a in the analysis Note that the data are missing if the average outcome when assigned to control differs between and if is for even This is simply to that use effectiveness and method effectiveness are not even the within of a The assumption may be to justify However, it may be less to justify within of a that with of can outcome treatment only through a causal some other to the that are on such the will be Thus, a is within to the A setting can be used to the same with a are central to the methods discussed in the next section for with dropout. it is possible to make in the of dropout without on or For example, if dropout and noncompliance are the same is, either comply and or comply and and if or a can be then validly estimates Rubin that the assumption, and the additional assumption that dropout for is not related to assignment, a valid estimator for can be are not always this section discusses a more than either of the It as on a to the nonignorable data into MAR data, just as a is used to data into data in the of the outcome so that the missing data difference between the and the missing are MAR, and one can as using the observed data by the This is here for a full based analysis the probability model for the missing data in a more way, and the problems with for example, by Rubin . one cannot on having a baseline enough to the missingness within (and hence MAR in the data set as a In the data in longitudinal studies can in this and the rest of this discussion therefore focuses on such data. A longitudinal study is one in which of are scheduled to be observed on each and the to be between assignment is a function of those In the case, and the one the outcome variable is simply the last value of the For example, if a severity is out and at after the outcome might be the at a then that there is a time for each individual such that that all are to and including that but The value of Ri then depends on the relationship of to if the individual not drop if he The longitudinal design that the outcome is the last in an individual longitudinal Let the actually observed, and denote those that are missing. to estimation valid in the face of nonignorable missingness is to if a model for the missing is by and how such a model can is if we as at the of this that the missingness will be with by in (but see the missing values with values and then proceeding with the analysis using as if it were the actual data Clearly, if each an of then the analysis using of it may be less than that will be The will depend on the that is the actually observed, and may as other can in principle be if missingness depends on the e.g. on a and not on its observed If it depends on the observed value e.g. on the stated not on an model for then the model will be if the condition however, if are the of the missingness can do is less than as the only missing data is the difference between the and the not the full value of the observation is a It the missing at time for individual i as the last observed value at time this does not make the missingness that over time a so that the last observed before is a biased of the last If the were a for a chronic disease, the would that a patient his fixed at the last observed value the fact that it usually in with the natural of the For most chronic there is no to that and disease would to the a patient out of a study were it out would be a The here certainly any causal estimator from the if the test treatment the of and if a point dropout, then more dropout will in the placebo group because their of is will these unobserved values to be the values last observed. this may make an analysis using more to than if it estimates would lead to of a neither the nor the of the can always be and in any the is not valid for method that scientific are the only ones that can They might be as simple as using the that is to justify missing values from a of the observed data on time for each who or they might be as as a full model, in an statistical model The key point is that, the of the model for the missing data cannot be on the observed for must therefore rest on the that them. This review with the observation that method effectiveness – average outcome difference due to taking a test treatment vs a control – is an important causal on which to decisions whether to use the treatment, and if at what The to estimator of treatment effect validly estimates use effectiveness (but only if any missing data are an of value for decisions decisions and whether to the use of the not method if both with the same However, trials may be by deviations from protocol, notably some patients to comply with the prescribed treatment, and the same or other patients out before the study can be observed. Both of these deviations that the intention-to-treat estimator no validly estimates method effectiveness (and in the case of dropout, it may not even validly use Depending on study other estimators may be that validly important method effectiveness It should therefore be a to design studies so that the such estimators to be valid are In general, other than the of randomized and assignment, and of complete the additional design the that one of the method effectiveness estimators discussed herein will be (1) treatment is the control treatment is of and the test treatment is only to those for it is is in all patients for the of the The study is and the is a function of the if those compliance with test treatment) are at baseline and In for data by deviations from protocol, scientific model-based analyses rather than simple of may be required to estimates of causal Such estimates will be sensitive to scientific assumptions that cannot be on the study data themselves. and assumptions should be and discussed in the study protocol to of
Author: Lewis B. Sheiner
Institution: University of California, San Francisco
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W1505057094
