Propensity score matching

In the statistical analysis of observational data, propensity score matching (PSM) is a methodology attempting to provide unbiased estimation of treatment-effects. The possibility of "bias" arises here because the effectiveness of a treatment may depend on characteristics that are associated with whether or not a participant in an observational study chooses, or is chosen, to receive a given treatment.

A treatment-effect is just jargon for the effect of something that is being studied -- like the consequences of smoking or the consequences of going to university. The people 'treated' are simply those -- the smokers, or the university graduates -- who undergo whatever it is that is being studied by the researcher. The language of 'treatment effects' comes originally from the medical literature where medical researchers have always hoped to isolate the true causal effects of different ways of dealing with disease. One way to do that is to run experiments.

In randomized experiments, the randomization enables unbiased estimation of treatment effects; for each covariate, randomization implies that treatment-groups will be balanced on average, by the law of large numbers. Unfortunately, for observational studies, the assignment of treatments to research subjects has been haphazard and not randomized; lacking randomization, observational studies frequently provide biased estimation of treatment effects and have imbalance on covariates.

In observational studies, the "treatment"-groups (or "exposure" groups) often exhibit imbalance on covariates. This covariate imbalance is confounded with treatments: It is difficult to attribute differences in responses to the "treatment" or "exposure" because the covariates are also believed to influence the response. The propensity score matching attempts to reduce the confounding effects of covariates, and so allow differences of responses to be attributed to differences of treatments (exposures).

Researchers try to decide how the world works, and, in particular, what causes what. To do this properly, it is not enough to observe correlations. It is necessary to try to understand causality. In other words, even when it is seen that smoking and cancer tend to occur together in a sample of human beings, the basic questions remain:

• is the smoking itself having a causal effect, or could it just be that cancer is caused by, say, poor diet and those who smoke tend not to eat healthy foods, or
• is some gene that leads to cancer and also by coincidence that that gene increases a person's enjoyment of cigarettes?

Similarly, people with university degrees tend later in their lives to earn more money than others without degrees, but is that because the education is actually causing the higher earnings?

Overview

PSM is for cases of causal inference and simple selection bias in non-experimental settings in which: (i) few units in the non-experimental comparison group are comparable to the treatment units; and (ii) selecting a subset of comparison units similar to the treatment unit is difficult because units must be compared across a high-dimensional set of pretreatment characteristics.

In normal Matching we match on single characteristics that distinguish treatment and control groups (to try to make them more alike). But If the two groups do not have substantial overlap, then substantial error may be introduced: E.g., if only the worst cases from the untreated “comparison” group are compared to only the best cases from the treatment group, the result may be regression toward the mean which may make the comparison group look better or worse than reality.

PSM employs a predicted probability of group membership e.g., treatment vs. control group—based on observed predictors, usually obtained from logistic regression to create a counterfactual group. Also propensity scores may be used for matching or as covariates—alone or with other matching variables or covariates.

History

In 1983, Paul Rosenbaum and Donald Rubin published their seminal paper that first proposed this approach.

General procedure

1.Run logistic regression:

• Dependent variable: Y = 1, if participate; Y = 0, otherwise.
• Choose appropriate conditioning (instrumental) variables.
• Obtain propensity score: predicted probability (p) or log[p/(1 − p)].

2.Match each participant to one or more nonparticipants on propensity score:

• Nearest neighbor matching
• Caliper matching
• Mahalanobis metric matching in conjunction with PSM
• Stratification matching
• Difference-in-differences matching (kernel and local linear weights)

3.Multivariate analysis based on new sample

• Use analyses appropriate for non-independent matched samples

Requirements for a good PSM

• Identify treatment and comparison groups with substantial overlap
• Use a composite variable—e.g., a propensity score—which reduces group differences across the covariates.

Disadvantages

Shadish, Cook, & Campbell (2002)^{[citation needed]} identifies further shortcomings with PSM:

• Large samples are required
• Group overlap must be substantial
• Hidden bias may remain because matching only controls for observed variables (to the extent that they are perfectly measured)

Pearl (2009) alerts to the following phenomena with PSM, as well as any matching method:

• Hidden bias may actually increase because matching on observed variables may unleash bias due to dormant unobserved confounders.
• Bias reduction can only be assured (asymptotically) by modeling the qualitative causal relationships between treatment, outcome, observed and unobserved covariates.

In general risks of PSM include:

• They may undermine the argument for experimental designs—an argument that is hard enough to make,
• They may be used to act “as if” a panel survey is an experimental design, overestimating the certainty of findings based on the PSM.

External links

• Introduction to Propensity Score Matching: A New Device for Program Evaluation. Lecture notes 2004
• Implementing Propensity Score Matching Estimators with STATA. Lecture notes 2001