Brian Doherty on Connecticut's Background Check Law and Murder Rate

Last week this blog shared Dr. John Lott's thoughts on that new study in the American Journal of Public Health, that purports to show a 1995 tightening in Connecticut's gun permit laws led to a 40% reduction in gun homicides over the next decade.  Now Reason.com is out with a new article investigating that same study. Many gun ban advocates immediately touted the study's finding to bolster their cause, but Reason.com's  Brian Doherty discovers that the study may not actually deliver what is advertised.

Given the amazingly complicated set of causes and incentives feeding into any human decision—and every gun homicide is the result of a human decision—establishing that the change in background check laws that "led to" a reduction in gun homicides "caused" them (even in that one Connecticut case, much less concluding that such laws can be relied on to have that effect in other places and times) is likely beyond any final authoritative conclusion via the usual methods of the social sciences.

Doherty details five specific problems with the study:

1. How do we know that synthetic-Connecticut really is a good marker for real Connecticut? The weight of that point seems to be almost entirely a pure case of believing that "past performance guarantees future results." Without saying anything about why it was so or should be presumed to always be so, the authors note that in the past Rhode Island's gun homicide levels matched Connecticut's very closely.
2. To return to the "appear" mentioned above in "Permit-to-purchase laws...appear to reduce the availability of handguns to criminals," given that we are assuming that the law is having all sorts of powerful effects on behavior and outcomes, don't we need to know something about how extensively or effectively the laws are being enforced, and have some decent data or reasonable guesses to be sure that the law's existence almost certainly is preventing many, many gun purchases by murderers that would have occurred without the law?
3. The authors are sure their gun-related cause leads to a gun-related effect by noting that the effects on homicide rates they allege to have found are almost all in gun homicides, not in other homicides. Curiously to me, the synthetic-Connecticut used to compare the non-gun homicides is very different than the mostly-Rhode Island one used for gun homicides; it is mostly New Hampshire. That comparison seems to be apples-oranges, and one wonders what the results would have been if they'd used the same synthetic Connecticut for both comparisons.
4. The study traces changes from 1995 to 2005; when I asked the CDC to send me the raw data numbers that the study relied on, the CDC warned me that "the coding of mortality data changed significantly in 1999, so you may not be able to compare number of deaths and death rates from 1998 and before with data from 1999 and after." [UPDATE: In an email sent after this post went up, the CDC says that "the change in...coding has almost no effect on homicide or suicide unlike other causes of death." So this point seems to be of little relevance.]
5. The study stops looking for effects 10 years after the law went into effect. Why might that be? Six of the eight years since 2005 for which CDC had data show Connecticut with a higher real gun homicide rate than 2005, the year that the authors chose to stop. If they had gone out to 2006, the reduction in rates in real Connecticut from 1995 to 2006 is cut to 12 percent.

Doherty discusses each of the problems in more detail and does a great job of picking the study apart.  It's a good read.