Computer controls and conclusions

Donohue and Levitt are somewhat famous for their bold claim, published in the May 2001 edition of the Quarterly Journal of Economics, that legalized abortion has reduced crime.

The Economist just put forward an amusing update that discusses a Federal Reserve Bank of Boston working paper and counter-claim that is based on a re-test of the data and analysis of the computer code used by Donohue and Levitt:

Messrs Foote and Goetz have inspected the authors’ computer code and found the controls missing. In other words, Messrs Donohue and Levitt did not run the test they thought they had—an “inadvertent but serious computer programming errorâ€?, according to Messrs Foote and Goetz

Fixing that error reduces the effect of abortion on arrests by about half, using the original data, and two-thirds using updated numbers. But there is more. In their flawed test, Messrs Donohue and Levitt seek to explain arrest totals (eg, the 465 Alabamans of 18 years of age arrested for violent crime in 1989), not arrest rates per head (ie, 6.6 arrests per 100,000). This is unsatisfactory, because a smaller cohort will obviously commit fewer crimes in total. Messrs Foote and Goetz, by contrast, look at arrest rates, using passable population estimates based on data from the Census Bureau, and discover that the impact of abortion on arrest rates disappears entirely.

I look forward to the question of this programming “error” being addressed by Donohue and Levitt. It does not seem to refute the premise of their conclusion outright as much as question the methodology and provide an opportunity to fix a control and re-run the tests themselves.

The big question, of course, is still whether there are controls that have a direct relationship to reducing crime and at what cost.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.