Ccsr.ac.uk

Causality—John Ermisch’s contribution to the panel
My discussion focuses on causal inferences in social science research using observational micro-data. The basic difficulty in identifying a causal effect of an individual’s action (or the experience of some event) on an outcome is that we do not know what the outcome for that individual would have been if they had not taken the action (did not experience the event). For instance, suppose our objective is to estimate the causal impact of a teenage birth on the child’s birth-weight. We do not know what the birth-weight of a woman’s first-born would have been if she had postponed her first birth beyond her teens. If we knew this ‘counterfactual outcome’ we would only need to compare the actual outcome with the counterfactual one, and that would be the causal effect. But of course, the counterfactual is inherently unobservable. Issues of estimation of the causal effect from observational data revolve around whose behaviour might serve as an estimate of this counterfactual, or who might be an appropriate ‘control group’. It is reasonable to expect that the causal effect of interest is different for different individuals. Thus, we are only able to estimate the average causal effect. In our example, the average difference in birth-weight resulting from a teen-birth compared to a later first birth. It is usually the case that the only average causal effect that we can identify with non-experimental data is the average impact of an action (event) among those individuals taking that action (experiencing that event), often called the effect of treatment on the treated. This measure can often be of considerable interest. For instance, in our example, policies that seek to reduce teenage childbearing are likely to target women who would have become teen mothers in the absence of the policy. Comparing the average outcome of individuals who took the action with that of individuals who did not usually does not identify the average causal effect among those who took the action, because even if they did not take the action, persons doing so may differ in mean outcomes from people who did not take the action. For example, women having teen-births may have had a lower birth-weight child even if they postponed childbearing until later—there is a ‘selection effect’ into the population of teen-mothers. Addressing this problem requires some minimal assumption or restriction that cannot be tested formally—it must be justified by a priori argument, outside evidence, substantive theory or some other informal means. A common approach is to find some variable(s) that affects the probability of taking the particular action, but does not directly affect the outcome (nor is it correlated with unobservable influences affecting the outcome)—often called ‘instrumental variables’, or just ‘instruments’. The existence and choice of instruments are the critical issues in the estimation of causal relationships. In our example, having a miscarriage as a teenager is such an instrument if having a teen-miscarriage does not directly affect the subsequent birth-weight of a woman’s first-born and the probability of a teen-miscarriage is not affected by unobserved influences that also affect birth-weight. A teen-miscarriage clearly affects the probability of having a teen-birth because a miscarriage precludes a teen-birth (ignoring the relatively few cases of both births and miscarriages as a teenager). While there is evidence that most miscarriages are random, resulting for example from abnormal formation of fetal chromosomes, epidemiological studies have found that smoking and drinking during pregnancy significantly increase the probability of a miscarriage, and this behaviour also affects birth-weight. Thus, if the available data does not have information on smoking and drinking during pregnancy, then a teen-miscarriage may not be a valid instrument. Another approach uses variation in actions within a family as an implicit instrument (e.g. comparing sisters, one of whom had a teen-birth while they other did not). Although this approach eliminates family influences on actions that are shared by siblings, the siblings may differ in other ways that are associated with outcomes. Again, other evidence or theory must be introduced to justify that the sibling-differences in actions is not affected by influences that also affect outcomes. In most cases, because of heterogeneity in causal effects, a particular instrument only estimates the average causal effect for those individuals whose behaviour is changed by the instrument. For instance, a policy raising the minimum school leaving age from 15 to 16 would serve as a valid instrument to estimate the average causal effect of an additional year of education on subsequent earnings for individuals who would have left school at 15 in the absence of the policy change. This raises the issue of the trade-off between what is called internal validity and external validity. In short, internal validity holds when the instrument is valid, or in an experimental context, when there is valid randomization, no contamination of control and treatment groups and no biasing attrition. External validity holds when the estimate of the causal effect generalizes to a broader population than that affected by the instrument or that in the experiment. The change-in-minimum-school-leaving age instrument is internally valid, but it would not necessarily reveal the earnings return to remaining in school one more year beyond 16. On the assumption that all miscarriages are random, the teen-miscarriage instrument is valid for estimating the effect of postponing childbearing on birth-weight. It is, however, conceivable that postponement of childbearing for some other reason, say because of the availability of more jobs in the local labour market, may affect birth-weight differently from postponement because of a miscarriage. In the sibling-difference approach, can we extrapolate the estimates from siblings who differ in the timing of their first birth to a wider population? In estimates of causal effects from valid experiments, how close do the external conditions have to be to the experimental conditions in order to use the results in these other contexts? In every case, knowledge of the channel(s) of the causal effect is very helpful in assessing the external validity of the estimates. Substantive theory and other evidence usually need to be brought to bear on the assessment of external validity, as well as on making a credible case for internal validity. In sum, caution is the byword. Certain types of causal effects may be unknowable, and knowledge of others usually requires consulting a range of evidence from different studies, based on different approaches to estimating the causal effect of interest (i.e. different ‘identifying assumptions’). While internal validity is obviously important, there is, as Robert Moffitt recently wrote*, ‘a danger in maximizing internal validity at the expense of external validity. To do so would lead to a field consisting only of very narrowly defined exercises without generalizability and to a collection of miscellaneous facts that do not add up to any general knowledge.’ Good social science requires due attention to both types of validity, and through its assessment of these, its substantive theory can help in the identification of causal effects that are useful to scientific development and policy. *R. Moffitt, Causal analysis in population research: an economist’s perspective, Population and Development Review, 29(3):448-458 (Sept. 2003).

Source: http://www.ccsr.ac.uk/methods/festival2004/programme/Fri/am/D/documents/ermisch_000.pdf