Study: Removing ‘bad apples’ from police forces unlikely to significantly reduce use-of-force complaints


The idea that a small number of “bad apples” are responsible for an outsized share of complaints against police officers has gained considerable traction over the last four decades. A new study considered the extent to which police misconduct is likely to be reduced by removing police officers identified early in their careers as being at risk for misconduct. The study concluded that replacing the top 10 percent of police identified as being the most likely to generate use-of-force complaints with officers who have not or are less likely to do so would reduce use-of-force complaints by just 6 percent over a 10 year period.

Conducted by researchers at the University of Pennsylvania and Princeton University, the study appears in Criminology Public Policy, a publication of the American Society of Criminology.

“Our analysis suggests that removing predictably problematic police officers is unlikely to have a large impact on use-of-force citizen complaints,” explains Aaron Chalfin, assistant professor of criminology at the University of Pennsylvania, who led the study. “Moreover, predicting who the problematic officers will be is difficult, so a better idea is to design early warning systems to deter problematic behavior and promote greater accountability.”

Analyses of police departments across the United States suggest that a small share of officers accounts for a large share of complaints about misconduct against police. A common estimate is that the top 2 percent of officers account for about 50 percent of known misconduct. Such statistics suggest that reform efforts should focus on terminating the “bad apples” but, as the authors show, such a computation is misleading.

In this study, researchers used data from several sources, including: 1) citizen complaints that implicated Chicago police officers between 2012 and 2017 and 2) use-of-force data from the Chicago Police Department’s tactical response reports from April 2011 to April 2016, focusing on the 11,283 officers employed by the department as of September 2017 and going back five years for each officer. The data were made available to the public by the Invisible Institute’s Citizens Police Data Project, which hosts a collection of nearly 250,000 complaints against Chicago police officers filed since 1988. The study followed Chicago police officers hired between 2000 and 2007 for 10 years, ranking officers by the number of complaints they received early in their careers to predict future risk of having a complaint filed against them.

Researchers found that between September 2012 and September 2017, 2,885 complaints against Chicago police officers involved use of force. Looking backwards, the top 10 percent of officers accounted for 70 percent of the complaints, leading many observers to posit that the Chicago Police Department could appreciably reduce use-of-force complaints by removing a small number of “bad apples.”

To determine whether this would be the most likely outcome, researchers carried out a policy simulation: First, they identified high-risk officers using information generated during their early-career 18-month probationary period, what the researchers termed an early warning system. Then they determined that there was considerable persistence in complaints over an officer’s career, suggesting that, on average, officers identified as high risk early in their career persisted in being characterized as high risk later.

Next, the researchers simulated replacing the high-risk officers with officers less likely to have use-of-force complaints (using a variety of different configurations of officers) to estimate the share of citizen complaints over a 10-year period that would be abated solely by terminating officers at high risk of use-of-force complaints.

The study estimated that removing the top 10 percent of the police force (a very difficult task since current rates of termination are approximately 0.2 percent annually) with officers drawn from the middle of the distribution of officers would reduce total complaints just 4.6 percent and use-of-force complaints 6.1 percent. These estimates, which are rather small, reflect the difficulty of identifying “bad apples” early in officers’ careers and suggest that a focus on computations that identify “bad apples” looking backwards (e.g., the top 10% of officers end up accounting for 70% of complaints) present a misleading view of the problem in policing.

The effects of terminating the top 10 percent of high-risk officers based on officers’ rankings of likelihood using force during a five-year probationary period (instead of the 18-month probationary period) were larger, at 16 percent. But the authors caution that terminating such a large number of officers after five years of service would be politically challenging.

“Early warning systems that simply identify problematic officers and incapacitate them, either through termination or reassignment, are unlikely to lead to large reductions in the use of force,” suggests Jacob Kaplan, a postdoctoral fellow at Princeton University, who coauthored the study. “But early warning systems that are coupled with rigorous oversight and genuine accountability might have a larger effect by generating deterrence or spillover effects among officers who are unlikely to be flagged as being high risk, or by changing departmental culture.”

What is needed in police departments, the authors say, is broad-based measures to improve managerial practices and increase accountability. Toward this end, they suggest that policymakers provide incentives for better and more complete reporting and discovery of police misconduct.

Among the study’s limitations, the authors note that while police departments’ early warning systems are not a panacea, they could produce more changes in use-of-force complaints above and beyond the estimates of this study. In addition, they note that their simulation did not identify the promise of early warning systems more generally or what the effects might be at scale.

Furthermore, the authors note that in a city like Chicago, even a small proportional reduction in use-of-force complaints can translate into hundreds fewer complaints annually. This, in turn, could save millions of dollars in settlements of lawsuits as well as improve relations between police and the community.

###