American policing is in the midst of a challenge to its legitimacy. The police killing of Michael Brown in Ferguson, MO in the summer of 2014 led to a firestorm of social media attention focused on police use of force against minority citizens. Social media and cell phone video fueled the viral spread of similar incidents across the United States in months to come, making police shootings a national (and international) conversation rather than one constrained locally to the jurisdictions where specific incidents occurred.

Rather than speculate about the impact of so important an issue, solid research should guide our understanding and policy responses.

Ferguson and related incidents resulted in civil unrest, microscopic scrutiny of police behavior, lawsuits, and officer terminations. Websites where citizens could post cell phone video of police-citizen interactions gained popularity, such as Cop Block and Reddit’s Bad Cop No Donut. This led some commentators, law enforcement officials, including the FBI Director, and politicians to warn the American public of an impending crime wave. More crime was argued to be the result of officers withdrawing from their duties out of fear of being on the next viral video.

The term given to this phenomenon is the “Ferguson effect.” It speaks to both “de-policing” (a reduction in proactive police strategies) as well as crises in citizen perceptions of criminal justice system legitimacy (as citizens become aware of injustices and emboldened by hypercriticism of law enforcement).

What does science tell us about this purported Ferguson effect? For the most part the debate has been based on anecdotal evidence and back-of-the-napkin data analysis. Heather Mac Donald of the Manhattan Institute was one of the first to use data to examine the Ferguson effect. She found increases in violent crime in a small group of cities and concluded that this was evidence of a nationwide crime wave on the horizon. Other findings have been reported by the New York Times, 538.com, the Washington Post, and the Brennan Center for Justice, which reveal some evidence of increases in violent crime.

The question that wasn’t answered by Mac Donald, and others, is whether increases in violence in the United States were a result of the events surrounding Ferguson.

We recently published the most rigorous study of the Ferguson effect on crime rates to date based on monthly crime data from 81 of the 105 largest U.S. cities (population over 200,000). We examined crime trends in the 12 months before and after Ferguson. The short answer: no nationwide crime wave could be pinned to Ferguson, at least among the largest U.S. cities. These results apply to overall crime rates, violent and property crime rates, and six out of the seven FBI Part I crimes.

However, there was a significant increase in robbery rates across the United States that began about the time that Michael Brown was killed in Ferguson. This is an important finding. It suggests that a Ferguson effect may exist for robbery—a violent street crime that can be effectively combated by good policing (or allowed to increase if de-policing is occurring).

A handful of cities—those with historically high levels of violence, a greater proportion of African-American residents, and socioeconomic disadvantages (e.g., Baltimore, St. Louis, New Orleans, and Detroit)—experienced increases in homicide rates after the Ferguson incident. Indeed, this is evidence of a Ferguson effect. It is notable however that each of these cities has been the subject of federal scrutiny and in two cases (New Orleans and Detroit) the police department has operated under a consent decree providing federal oversight of police operations.

Our study didn’t please everyone. One of the more common critiques levied against our work was that we incorrectly conclude that there is no Ferguson effect on crime. Heather Mac Donald in City Journal, for example, in response to our findings, held: “the existence of a Ferguson effect does not depend on its operating uniformly across the country in cities with very different demographics.”

We agree with Mac Donald. As would be expected, we observed heterogeneity in how cities responded to the events in Ferguson—most cities experienced no change in crime rates while a small number saw increases. This seems straightforward enough. The analogy would be a stock portfolio, where some holdings increase even in a down market. We concluded that some cities could have been “primed” for a Ferguson effect on crime, a conclusion that Mac Donald curiously termed “groundless.”

But we wonder why Mac Donald titled her original opinion piece (the one that really stoked the Ferguson effect fire) in the “The New Nationwide Crime Wave” (our italics). Now strong empirical evidence exists that finds no evidence of a nationwide crime wave among large cities. Despite this, the knee jerk reaction is a revision of the original hypothesis out of fear that the facts and good research will get in the way of a good story, or a political opinion. Revising theories in the face of observation is part of the scientific process. The only difference is that Mac Donald doesn’t acknowledge that her original hypothesis was wrong. And our results are good news for cities and the police: on the whole crime is not up.

A second criticism offered by Mac Donald and others focused on our use of an “arbitrary” cut point to determine statistical significance (p < 0.05). While not an absolute standard, this level of significance is the most commonly used in the social sciences. Let’s assume that we ignored this convention and instead drew conclusions based on numbers that failed to recognize the possibility of chance fluctuations in crime. We would have been rightly criticized for fishing for effects to prove a point. That’s not how science is supposed to work.

One alternative is to look away from statistical significance and focus on substantive significance—that is, effect sizes. What was the magnitude of the effect of Ferguson on crime trends? When you examine the effect size for violent crime we see that it is expected to change by 0.34 offenses per 100,000 residents per month after Ferguson. This suggests that if a Ferguson effect on violent crime exists, we would observe an average of 4 more violent offenses per 100,000 people over the course of an entire year in the 81 U.S. cities. When we standardize this effect—placing it on a comparable metric—we find a 0.008 post-Ferguson monthly redirection in violent crime. Over a year, that’s a 0.096 standard deviation increase. An effect size of 0.20, according to Jacob Cohen, the pioneer of effect size interpretations, is “small.” In short, this isn’t an effect size that proponents of a Ferguson effect would want to trumpet as evidence in support of their claims.

Mac Donald seizes on Fordham law professor John Pfaff’s ill-informed tweets about a “tenfold” increase in violent crime after Ferguson. Yes, the rate of change in violent crime was 10 times greater after Ferguson. But that’s because the violent crime rate trend before Ferguson was practically flat. Any change, whether positive or negative, would have appeared massive by that standard. An analogy may be in order. Let’s say you are walking on a flat street at sea level and then step up a curb. Your elevation has now changed drastically (0 feet above sea level compared to 4 inches above sea level on the curb). Does this mean you are now standing on a mountain? No, it does not.

Ferguson Effect Detractors Are Wrong

Sydney. London. Toronto.

QuilletteHeather Mac Donald

This does not imply that there is no such thing as a Ferguson effect. Indeed, the evidence points to other possible Ferguson effects. One study just published in Law and Human Behaviour showed that a sizable portion of sheriff’s deputies indicated that they have become less motivated in recent months as a result of negative publicity surrounding law enforcement. Indeed, this is evidence of a Ferguson effect on officer morale and behavior. The good news, however, was that there was no indication in the study that such sentiment translated into de-policing in the form of a withdrawal from community partnerships.

Another study published in Justice Quarterly revealed that officers who felt less motivated as a result of negative publicity surrounding their profession were significantly less likely to have confidence in their own authority as cops. Again, this provides support for a Ferguson effect. The bigger questions are whether and how such negative publicity translates to police activities on the street.

The Ferguson effect may mean different things. As a consequence there may be no simple yes or no answer to questions about its effect. Ignoring the nuances in a debate about crime and criminal justice leads to bad policy and potentially negative outcomes.

Despite this, some insist that a Ferguson effect exists and that crime rates are increasing. They point to unfair scrutiny of the police as a cause. Why anyone would want a Ferguson effect to exist, and criticize research of contrary evidence, is an interesting question. This tells us that such individuals believe that police are not professional enough, not trained well enough, and too hesitant under pressure to withstand the new reality that their actions can be caught on camera. This sentiment seems to be particularly “anti-cop.” It seems to us that it is much more “pro-cop” to conclude that a vast majority of officers are well-trained professionals who can withstand pressure from public scrutiny. If one accepts this premise, we would certainly not expect large groups of officers to de-police and cause higher crime rates in our communities.

Maybe herein lies the problem—threatening people with the prospect that violent crime will increase if they protest police behaviors suggests that the police should not be accountable for their behavior. The recent events in Richland County (Columbia), South Carolina involving the video of a Richland County Sheriff’s Deputy tossing a teenage girl across a classroom is an example of how a law enforcement executive can take the opposite approach to this prevailing opinion. In response to this, Sheriff Leon Lott initiated a quick, transparent, and professional investigation, fired the deputy for his actions, and people in the community largely moved on with their lives. No violent crime increase occurred, no violent protests took place. There was no “Ferguson effect” in an area that we may be most likely to see it—an urban jurisdiction with racial diversity, higher than average crime rates, and in the South which is a region that apparently has had a history of racial tension.

Any assessment of a Ferguson Effect should be one of fact and not ideology. Yet some insist that academics are out to satisfy a progressive agenda. Such attacks have no grounding in the facts and themselves advance an ideology. When such opinions come from academics it tells us that they can’t accept the evidence and that they instead resort to ad hominem attacks. Indeed, police leaders across the country are using data and research to better understand their crime problems and craft more effective responses.

In the end, one thing about the Ferguson effect is crystal clear. A substantial segment of the American public is questioning the legitimacy of police actions, including the use of force. This attention is a Ferguson effect in itself.

Perhaps we should spend more time worrying about the legitimacy crisis rather than a Ferguson effect on crime. Rather than warning the public to stop criticizing police we should work with them to restore trust and legitimacy. This argument does not give carte blanche to rabble rousers who aim to exact violence against police and file false complaints. On the contrary, such behavior is not acceptable.

At the same time, law enforcement, politicians, and talking heads need to stop threatening citizens by telling them that the police are going to de-police unless they stop criticizing officers. Such statements only communicate to the public that we should not hold public servants accountable. Such unscientific rhetoric may appeal to many but serves no one. What we need is more good inquiry and less speculation. We invite others to join the debate, but with data and analysis.