How Much Should We Trust Differences-In-Differences Estimates?

Bertrand, B. and Duflo , E. and Mullainathan, S. (2003) How Much Should We Trust Differences-In-Differences Estimates? The Quarterly Journal of Economics . pp. 1-32.

PDF - Accepted Version
| Preview


Most papers that employ Differences-in-Differences estimation (DD) use many years of data and focus on serially correlated outcomes but ignore that the resulting standard errors are inconsistent. To illustrate the severity of this issue, we randomly generate placebo laws in state-level data on female wages from the Current Population Survey. For each law, we use OLS to compute the DD estimate of its “effect” as well as the standard error of this estimate. These conventional DD standard errors severely understate the standard deviation of the estimators: we find an “effect” significant at the 5 percent level for up to 45 percent of the placebo interventions. We use Monte Carlo simulations to investigate how well existing methods help solve this problem. Econometric corrections that place a specific parametric form on the time-series process do not perform well. Bootstrap (taking into account the autocorrelation of the data) works well when the number of states is large enough. Two corrections based on asymptotic approximation of the variance-covariance matrix work well for moderate numbers of states and one correction that collapses the time series information into a “pre”- and “post”-period and explicitly takes into account the effective sample size works well even for small numbers of states.

Item Type: Article
Author Affiliation: University of Chicago Graduate School of Business, National Bureau of Economic Research, and Center for Economic Policy Research
Subjects: Social Sciences > Agricultural Economics
Divisions: General
Depositing User: Mr Daneti Raju
Date Deposited: 27 Aug 2014 06:00
Last Modified: 27 Aug 2014 06:00
Official URL:

Actions (login required)

View Item View Item