A few weeks ago Mother Jones, having not learned the lesson of their absurd article claiming mass shootings are on the rise, published a list of 10 Myths about guns and gun control from Dave Gilson. And I’m going to debunk their debunking again because the article represents what I believe is one of the worst sins in the field of Mathematical Malpractice: cherry-picking. As I went through this, it became obvious that MJ was not interested in the facts, really. What was motivating them was the argument. And so they picked any study — no matter how small, how biased or how old — to support their point. They frequently ignore obvious objections and biases. And they sometimes ignore larger more detailed studies in favor of the smaller ones if it will support their contention.

We see this a lot in the punditocracy, unfortunately. As Bill James said, most people use studies the way a drunk uses a lamppost — for support, not illumination. In any sufficiently advanced but difficult field of study, you will find multiple studies examining an issue. Let’s say it’s a supposed connection between watching Glee and having a heart attack. If there is, in reality, no connection between the two, you might find eight studies that show no connection, one that shows an anti-correlation and one that shows a correlation. This is fine. This is science. There are always outlier studies even if all the researchers are completely ethical and honest. The outliers fall away when your interest is the question and you look at all the evidence. But the outliers dominate the discussion from those who have an agenda.

This happens a lot in the gun debate. On both sides, really. But Mother Jones’ article is a particularly putrid example of this because that’s basically all it does: collect the cherry-picked nonsensical studies that support their anti-gun agenda. It’s quite remarkable actually; almost a clinic in how not to do research.

But here’s the one thing that really tips you off. There is one myth that Mother Jones does not debunk. It’s a myth that’s really independent of what you think of gun ownership … unless you’ve already staked part of your reputation and agenda on the myth that gun violence is increasing. In fact, all forms of violent crime have been falling for twenty years. This is, in my mind, the single most important fact in debates over crime and violence and the single most important myth to debunk.

MJ does not address this myth. They don’t even talk about it. That is a huge tell.

Myth #1: They’re coming for your guns.

Fact-check: No one knows the exact number of guns in America, but it’s clear there’s no practical way to round them all up (never mind that no one in Washington is proposing this). Yet if you fantasize about rifle-toting citizens facing down the government, you’ll rest easy knowing that America’s roughly 80 million gun owners already have the feds and cops outgunned by a factor of around 79 to 1.

Maybe we can agree that this is a myth. On the other hand, when you have an anti-gun lobby that has (1) identified an unarmed society as their goal; (2) lauded nations that have banned their guns; and (3) advocated policies like restricting bullets that would make guns effectively useless, I think it behooves us to think they have that goal in mind.

(I also find it odd that this fact is often placed side-by-side with the “you’re not going to use an AR-15 to stop an Abrams tank” response to the idea of revolution. They need to make up their minds. Are we powerless against our military? Or do we outnumber them 79 to 1?)

Myth #2: Guns don’t kill people—people kill people.

Fact-check: People with more guns tend to kill more people—with guns. The states with the highest gun ownership rates have a gun murder rate 114% higher than those with the lowest gun ownership rates. Also, gun death rates tend to be higher in states with higher rates of gun ownership. Gun death rates are generally lower in states with restrictions such as assault-weapons bans or safe-storage requirements.

Problem: they’re looking only at gun deaths. That makes sense if you, like Mother Jones, believe that guns are an evil talisman that compels people to murder. But most people would think that the goal is to prevent death Moreover, looking at gun deaths includes suicides, which comprise two-third of gun deaths. There is some evidence that banning guns would lower the suicide rate; guns have a far higher suicide success rate (on the other hand, other methods of suicide are more favored by people making suicidal gestures who don’t want to really kill themselves).

I can’t embed the graphic but when you look at the total violence rate from all methods of killing — using the same sources they link — the correlation is not nearly as strong (R^2 of .13) The trend is 0.10 for every percent. So eliminating ALL guns — even if you assume that there is no increase in criminality — would reduce the death rate to about 14.8 or basically as peaceful as Iowa with its 44% ownership rate and Rhode Island with its 13%.

What’s more, there are significant outliers. Nevada and New Mexico are more violent than you would expect based on the linear trend. Nebraska, Iowa and Minnesota have high ownership rates but relatively low levels of violence. And there is one huge outlier that shatters the graph: the District of Columbia, which has both a lower gun ownership rate and a higher crime rate than any state. DC is an unusual case, of course. Violence tends to be concentrate in cities and DC is all city. That having been said, the official DC gun ownership rate is a minuscule 5%, half that of Hawaii, mainly due to the draconian anti-gun laws they had until recently.

The other problem this point runs into — and you’re going to see this again and again — is that correlation is not causation. Maybe guns do cause violence. But you could equally argue that being in a violent area makes you more likely to buy a gun for self defense.

What would make sense here is a longitudinal study, one that looks at how violent crime rates rise or fall when gun laws are liberalized. Mother Jones ignores this because the last twenty years have seen gun laws liberalized while crime rates have plunged. That doesn’t show that liberalized gun laws prevent crime, of course. John Lott claims they do; others are more mixed. The fall in crime in multi-variate and it’s difficult to tease out the effect of one policy (least of all 50).

My point, however, is that if you’re going to argue that gun ownership puts people in danger, this is the wrong data to use.

Myth #3: An armed society is a polite society.

Fact-check: Drivers who carry guns are 44% more likely than unarmed drivers to make obscene gestures at other motorists, and 77% more likely to follow them aggressively.
• Among Texans convicted of serious crimes, those with concealed-handgun licenses were sentenced for threatening someone with a firearm 4.8 times more than those without.
• In states with Stand Your Ground and other laws making it easier to shoot in self-defense, those policies have been linked to a 7 to 10% increase in homicides.

The first study is a self-reported study of 2400 drivers. It’s odd it is invoked since it recalls one of most conspicuous and inaccurate predictions of the gun-control advocates: that conceal-carry laws would create shootouts over car accidents. They didn’t. It also conflates correlation with causation. And it is frankly a bit pointless.

For the second study, I can only see the abstract. They did note that conceal-carry holders were less likely to be convicted of crimes but that their convictions were more likely to involves sexual offenses, gun offenses and offenses involving a death. There’s a bit of flim-flammery in that sentence, however and I can’t see the article to see if it’s born out. It seems to say that while gun owners are less likely to commit crimes, their crimes are likely to be more serious. What’s missing? Usually when something is stated that way, it’s to conceal that gun owners are less likely to commit crimes involving a death, gun or sex but slightly less less likely than they are to commit other crimes.

Back in this thing called reality, the Texas Department of Public Safety studied all crimes committed in Texas and found that less than 1% were committed by conceal-carry holders. That’s compared to about 2% of all Texans who have conceal-carry. Those results reflect the reality in other states as well.

The final study is problematic. If you look at the graphs they include, it’s clear that they’re looking at noise. But they then do a statistical analysis which has 9 dependent variables and and 11 control ones. This crosses me as a massive overfitting of the problem. What they show, at most, is that stand your ground states did not have the drop in crime in 2009 and 2010 that other states did. But the data are so noisy, it’s really hard to make that conclusion, especially when they, oddly, plot it in log space to conceal just how noisy the data are. It’s frankly bad science and crosses me as cherry-picking. I feel like Mother Jones did not look for the best study of this; they look for a study that supported their conclusions, no matter how faulty it was.

As I noted above, it’s very difficult to pick out the effect of CCL’s on violent crimes rates because crime has been falling everywhere. But this issues had been addressed in far more intelligent ways than three marginal studies.

Myth #4: More good guys with guns can stop rampaging bad guys.

Fact-check: Mass shootings stopped by armed civilians in the past 30 years: 0
• Chances that a shooting at an ER involves guns taken from guards: 1 in 5

This stinks. I noted before how their claim that mass shooting were never stopped by civilians was entirely a product of their selection criteria that basically eliminated all the mass shootings that were stopped by someone armed. They also ignore the deterrent effect that guns are supposed to have. John Allen Fox’s study shows that mass shootings have been flat over the last thirty years.

The second point comes from a study of 265 incidents in emergency rooms. I hardly think that’s a representative sample of anything. It’s so obscure, I have to believe it was cherry picked. Back in reality, I found this (PDF) 2001 report from the Justice Department which interviews tens of thousands of inmates. Most of them got their guns either illegally or from a friend. The number who got them from their victims was too small to be included.

I think this a perfect demonstration of how Mother Jones selectively cites their stats. The 2001 study is linked in Myth #10 to show that most criminals get their guns in private sales. But when it comes time to figure out how many get their guns off their victims, Mother Jones does not cite the massive study that shows very few guns are obtained that way. No, they go to an obscure study of 265 ER incidents.

Myth #5: Keeping a gun at home makes you safer.

Fact-check: Owning a gun has been linked to higher risks of homicide, suicide, and accidental death by gun.
• For every time a gun is used in self-defense in the home, there are 7 assaults or murders, 11 suicide attempts, and 4 accidents involving guns in or around a home.
• 43% of homes with guns and kids have at least one unlocked firearm.
• In one experiment, one third of 8-to-12-year-old boys who found a handgun pulled the trigger.

The first stat ignores the work of Gary Kleck, whose well-cited work estimated a couple of million defensive uses of weapons every year, about five times the rate of aggressive gun violence. Even if he overestimates, he is unlikely to have done so by a factor of 30. MJ basically commits one of the classic blunders of the anti-gun faction: only counting defensive uses of weapons when someone is killed or seriously injured. A warning shot, a waving around of a gun, chambering a round as you come down the stairs — things Kleck counts — are ignored. The potential deterrent effect — criminals being unwilling to invade a house where they are likely to encounter a gun — is ignored.

These effects are asymmetric. Defensive uses of weapons are less likely to be reported. People defending themselves are, by definition, less violent than attackers and therefore less likely to fire a gun. The deterrent effect is almost impossible to measure statistically. Almost all the biases in these studies go against the “guns are never used defensively” position. Mother Jones doesn’t even acknowledge this.

The second stat is interesting but not really relevant. Accidental gun deaths are thankfully rare despite all the unlocked weapons. The study is also garbage, or at least quoting it that way is. Looking at the study, only 9% of guns were kept unlocked and loaded, which is the really dangerous situation. Moreover, “unlocked” includes not having a trigger guard. So, according to the survey, my dad was in that category because he had his unloaded guns in a closet with a keyed knob, a deadbolt and top bolt. I’m in that category even though my gun is in a safe and I have no bullets. Once again, Mother Jones has selected the study that most supports their ideology and, apparently, only read the abstract.

The third stat is garbage. This was a study of 64 boys. They were placed in an observation room and told to play. Most of them thought the gun was a toy. I’ve got news for Mother Jones: most parents do not conceal guns in their children’s playrooms and then tell them to play with anything they find. Most of them warn their kids about guns. Putting them in that kind of an environment tells you nothing. And it is belied by the thankfully low number of accidental deaths. If you combine “fact” 2 with “fact” 3, we should have accidental shootings constantly.

Myth #6: Carrying a gun for self-defense makes you safer.

Fact-check: In 2011, nearly 10 times more people were shot and killed in arguments than by civilians trying to stop a crime.
• In one survey, nearly 1% of Americans reported using guns to defend themselves or their property. However, a closer look at their claims found that more than 50% involved using guns in an aggressive manner, such as escalating an argument.
• A Philadelphia study found that the odds of an assault victim being shot were 4.5 times greater if he carried a gun. His odds of being killed were 4.2 times greater.

Mother Jones is repeating themselves by this point. The first fact simply looks at crime stats and finds that killing over arguments are ten times as likely as justifiable homicides. Once again, most defensive uses of weapons do not involve a killing. Ironically, it is the liberal anti-gun Mother Jones who have formed their self-defense ideas from movies and television. And nothing, nothing in those statistics has any relation to gun ownership or conceal-carry. There is no indication whether the guns used to kill over arguments were legally owned or not (according to the study they cite later, most of them were obtained illegally). Even the raw statistics show the transparency of the argument. In a typical year, a couple of thousand people are killed in arguments. Even if we assume these are all legal gun owners (most of them aren’t), that it less than one in a hundred thousand weapons

The second study is jaw-droppingly dubious. It involved phone interviews and an evaluation of whether the gun was used defensively or offensively, often ignoring how the victim/perpetrator viewed the incident. No one except an ideological gun control advocate would think this was scientific. Moeover, even if you take the stats seriously, that means 1.5-3 million Americans did use guns to defend themselves. I hate to tell Mother Jones, but that statistic is pretty close to what Kleck found.

The third study is incredibly noisy. The confidence interval is that gun carriers are 1-17 times more likely to be assaulted. I’m also having trouble figuring out their stats, since their raw data doesn’t indicate nearly as strong a correlation. In fact, there’s very little correlation at all. There’s *much* more obvious disparities in alcohol and illicit drug involvement.

Myth #7: Guns make women safer.

Fact-check: In 2010, nearly 6 times more women were shot by husbands, boyfriends, and ex-partners than murdered by male strangers.
• A woman’s chances of being killed by her abuser increase more than 7 times if he has access to a gun.
• One study found that women in states with higher gun ownership rates were 4.9 times more likely to be murdered by a gun than women in states with lower gun ownership rates.

The first study is irrelevant. All people are more likely to be murdered by people they know. And, in general, women are not heavily involved in organized crime or drug dealing, which correlate with homicide incidents involving strangers.

The second study doesn’t support their point and they are misquoting it. It identifies previous abuse as by far the most important risk factor for women being killed by their partners. They do find a relationship to gun ownership, although a smaller one than previous studies. But if you want to keep women from getting killed, getting them away from abusive partners is, by far, the most important factor.

The third study mainly restates the earlier point on the correlation of gun violence to gun ownership; see correlation-causation. But MJ misquotes a study again. That statistic comes from a raw comparison of the five highest-gun ownership states to the five lowest. This is an incredibly dubious way of analyzing data, especially when you consider the states:

High-gun states: Louisiana, Arkansas, West Virginia, Mississippi, Alabama
Low-gun states: Hawaii, Massachusetts, Rhode Island, New Jersey, Delaware

I would submit that there are larger differences between those states than rates of gun ownership. I also don’t think it’s valid to measure things by comparing the most extreme elements. I would much rather trust my analysis of all 50 states.

Also of note — their data do not show that women are in any particular danger. Using the raw data from the earlier talking point, I find similar ratios for overall homicides. In fact, guns are involved in 2/3 of homicides according to the CDC. But, according to this study, they are only involved in about half of homicides where the woman is the victim. Doesn’t this suggest that guns aren’t the real problem?

And to be frank, all of these studies give me the opposite idea than Mother Jones. Women rarely own guns and rarely use them to defend themselves. Nevertheless, they can be victims. And half the time, their murder does not use a gun, but fists, knives or blunt objects. Doesn’t that indicate maybe they should own guns? That guns can be an equalizer? I don’t know. But I would suggest the question is more complicated than selectively quoting and misquoting three studies.

In any case, thousands of women disagree with these points.

Myth #8: “Vicious, violent video games” deserve more blame than guns.

I agree that this is a myth. No comment necessary.

Myth #9: More and more Americans are becoming gun owners.

I agree that this is a myth. No comment necessary.

Myth #10: We don’t need more gun laws—we just need to enforce the ones we have.

Fact-check: Weak laws and loopholes backed by the gun lobby make it easier to get guns illegally.
• Around 40% of all legal gun sales involve private sellers and don’t require background checks. 40% of prison inmates who used guns in their crimes got them this way.
• An investigation found 62% of online gun sellers were willing to sell to buyers who said they couldn’t pass a background check.
• 20% of licensed California gun dealers agreed to sell handguns to researchers posing as illegal “straw” buyers.
• The Bureau of Alcohol, Tobacco, Firearms, and Explosives has not had a permanent director for 6 years, due to an NRA-backed requirement that the Senate approve nominees.

There isn’t really a math issue here (although both of the studies contacted just over a hundred gun dealers). But I would editorialize that the “gun lobby” is more than just the NRA. In fact, Congress is likely to pass laws that will address points one to three and such ideas have broad support among gun owners. But I would add that points two, three and four basically agree with the supposed Myth. Straw sales bypass existing law. Online gun sellers, if they sell to people who fail background checks, are breaking the law (although very few criminals get their guns online). The lack of an ATF director is a failure to enforce the law. How, exactly, does this disprove the myth?

Between this and the “study” on mass shootings, Mother Jones has made their bias clear. They are not really interested in the facts. They are not interested in the data. They are interested in an agenda. They are interested in portraying gun ownership as a destructive evil force in society. And no matter how far they have to dig and how much they have to twist the data, they will find the “facts” to support this position.

That’s not unusual. People do that. But at Mathematical Malpractice Watch, I see no reason to let people get away with it. I see no reason to pretend someone is a serious contributor to a debate — and MJ is considered a serious contributor to the debate — when they frankly aren’t.