Key Takeaways
-
Peer-reviewed research (Ridgeway 2026, Journal of the American Statistical Association) confirms threshold-based EIS misses the officers who need support most because it counts incidents without weighing context.
-
Predictive EIS, built on University of Chicago research and validated across 80 million-plus officer performance records, reaches 85% model precision with an average 198-day lead time before adverse events. Threshold-based methodologies sit around 30%.
-
Agencies running First Sign® have seen a 13% reduction in use of force, a two-level drop in force severity, and a 48% decrease in citizen complaints, while arrest activity stayed stable.
A peer-reviewed study published in February 2026 in the Journal of the American Statistical Association confirms what police leaders have long suspected: threshold-based early intervention systems fail because they count incidents without accounting for context. The study, by Greg Ridgeway of the University of Pennsylvania, analyzed seven years of Seattle PD use-of-force data and found that officers whose threshold system would never flag were sometimes the consistent escalators, while officers with high force counts often acted proportionally to the situation.
The finding aligns with what Benchmark’s multi-jurisdictional database has shown for over a decade: roughly 5% of officers generate 66% of injuries and disproportionate use-of-force incidents. First Sign® Early Intervention, built in partnership with the University of Chicago, was designed to find that 5% by analyzing context, behavioral patterns, and over 90 model variables rather than raw counts. Agencies running First Sign® have seen a 13% reduction in use of force and a 48% decrease in citizen complaints, while arrest activity stayed stable.
If your agency uses an EIS that flags the wrong officers and misses the ones who need support most, this is why.
Why don’t threshold-based early intervention systems work?
A threshold-based EIS flags officers who exceed a fixed count of force incidents, complaints, or other events within a defined time window. The approach has been the field standard since Walker, Alpert, and Kenney first formalized the early warning system concept in Police Quarterly (2000). It’s also the architecture CALEA Standard 35.1.9 was originally written around. The logic is simple, and that simplicity is the problem.
Officers working in high-crime areas, on night shifts, or in specialized units like warrant service encounter more situations involving force. A threshold system treats all those encounters identically. The predictable result: threshold-based flags disproportionately identify the most active officers (false positives) while missing genuine risk among officers with lower activity levels (false negatives).
Ridgeway’s study highlights a distinction threshold systems miss entirely: the severity of force relative to the situation. An officer who consistently uses more force than peers in identical circumstances presents a very different risk profile from one whose force is proportional to the call. Benchmark’s patented approach to capturing force data was designed with this distinction in mind, emphasizing proportionality as a core measure. Threshold systems can’t make that distinction. They only count.
As Dr. Ugochi Jones, VP of Data Science at Benchmark Analytics, frames it: “Trigger-based early warning systems cast a broad net. They do not focus on the individual officers who truly need support.”
What is a predictive early intervention system?
A predictive early intervention system uses statistical models and machine learning to identify officers at elevated risk based on the full pattern of their work, not a single incident count. The methodology has academic origins. The foundational study by Helsby, Walsh, Ghani et al. (2018) in Criminal Justice Policy Review, conducted with the Charlotte-Mecklenburg Police Department, demonstrated that a machine learning model trained on multi-source officer data reduced false positives by 20% and increased true positives by 75% compared to the department’s existing threshold EIS. That work, recognized with the IACP’s 2018 Leadership in Law Enforcement Research Award, is the academic foundation of First Sign®.
What distinguishes a predictive EIS in practice is what it sees and how it reasons. First Sign® evaluates over 90 model variables across five-plus data sources (officer attributes, arrest history, use of force, internal affairs, CAD/RMS), validated against more than 80 million officer performance records spanning 2,500-plus agencies. It achieves 85% model precision, with an average lead time of approximately 198 days between the first signal and an adverse event. Six months is enough time for a meaningful intervention. A reactive alert delivered after the fact is not.
How does the Ridgeway study validate predictive EIS?
Ridgeway applied his model to 4,821 Seattle PD use-of-force incidents involving 1,503 officers across seven years. Instead of counting incidents, the model compares each officer’s force choices against peers present at the same scene. Because every officer on scene shares the same environmental context (location, subject behavior, lighting, conditions), the model mathematically eliminates those situational factors and isolates each officer’s individual tendency to escalate.
Out of 1,503 officers, the model identified nine with a statistically high probability of escalating force beyond what peers would use in identical circumstances. That’s less than 1% of the force. At the other end, it found 13 officers present at numerous force incidents who almost never used force themselves. That could reflect strong de-escalation skill or supervisory roles, but it could also signal hesitation when the situation demanded action.
One finding deserves attention from any agency running threshold rules. A flagged officer in the dataset had fewer than 10 incidents across seven years and never exceeded Level 1 force. No threshold system would ever have caught that officer. But the model showed that in nearly every incident, this officer was the one who used force while peers did not. Meanwhile, another officer with 15 Level 2 incidents and three Level 3 incidents was not flagged, because their force was consistent with what peers used in identical high-severity situations. Context changed the picture entirely.
What should an effective early intervention system do?
Five characteristics distinguish an effective EIS from a system that simply meets compliance minimums. Dr. Jones lays them out plainly:
- Timely. Flags officers before adverse events, with enough lead time for an intervention to take place.
- Adaptive. Recognizes that policies, community dynamics, and agency conditions change, and updates accordingly.
- Evidence-based. Incorporates research on policing techniques, environmental factors that affect officers, and the interventions that demonstrably help.
- Precise. Targets the right individuals. Does not cast a wide net that frustrates supervisors and stresses officers who do not need an intervention.
- Action-oriented. Makes the supportive next step easy. Identification without a clear path to action is a research finding, not an early intervention.
A threshold-based system can claim transparency and simplicity. It cannot credibly claim any of the five.
Comparison: trigger-based vs. predictive early intervention
| Dimension | Trigger-based EIS | Predictive EIS |
|---|---|---|
| Foundation | Predefined rules and thresholds | Statistical models trained on outcomes |
| Data scope | Single agency, limited history | 80M records, 2,500+ agencies |
| Accuracy | ~30% precision | 85% precision |
| Lead time | After incidents accumulate | ~198 days before adverse event |
| Adapts over time | No, rules are static | Yes, continuous retraining |
| Connects to action | Alert only | Embedded support plans, intervention tracking |
How does First Sign® apply this approach?
First Sign® translates the predictive methodology into supervisor-ready workflow. Where threshold systems generate an alert and stop, First Sign® connects directly to C.A.R.E.® (Case Action Response Engine), Benchmark’s research-based intervention case management platform. Supervisors build support plans inside the system, document outreach, assign accountable parties, and track whether the intervention reduces underlying risk over time. Each completed intervention feeds back into the model, so the system learns which support approaches work for which patterns.
The outcomes show up in the field. Across a subset of partner agencies employing nearly 10,000 sworn officers, First Sign® implementation has been associated with a 13% reduction in use of force, a two-level reduction in force severity, and a 48% decrease in citizen complaints, while arrest activity remained essentially stable. One agency saw 31% reductions in force severity, 72% fewer officer injuries, and 52% fewer citizen complaints during the observation period.
Independent research supports the broader pattern. Katz, Cheon, Freemon, and Wallace’s 2025 study in Police Quarterly on EIS effectiveness found that the predictive validity of officer identification meaningfully shaped intervention outcomes. The system has to flag the right officers. Otherwise, the intervention work that follows is misallocated.
Frequently Asked Questions
What is the difference between an early warning system and an early intervention system?
The terms are often used interchangeably. Early warning systems historically focused on detection. Early intervention systems focus on detection, plus the support response that follows. A predictive EIS combines both: it identifies risk earlier and connects directly to documented intervention plans.
Are trigger-based EIS still required by CALEA or DOJ consent decrees?
CALEA Standard 35.1.9 and most DOJ consent decrees require an early intervention system. They do not require a specific methodology. Agencies meet the minimum with threshold rules but increasingly move to predictive systems for performance and defensibility.
What Is a Data-Driven or Predictive Early Intervention System?
A predictive EIS uses machine learning trained on multi-agency data to identify officers based on patterns of behavior, context, and multiple data sources rather than raw event counts. First Sign® is the only research-based predictive EIS commercially available, developed in partnership with the University of Chicago and validated across the world’s largest multi-jurisdictional officer performance database.
Will officers see a predictive EIS as surveillance?
A well-implemented predictive EIS is a supervisory support tool, not a discipline tool. The lead time the model provides is what makes a non-punitive support response possible. Agencies that pair predictive EIS with clear policy on how alerts are used report higher officer trust than agencies relying on threshold rules with no support pathway.
Can we keep our existing threshold rules and layer a predictive system on top?
Yes, and most agencies do. Predictive systems do not replace policy rules. They identify officers the rules miss and reduce false positives among officers the rules flag unnecessarily. The rules become a floor. The predictive model is what gets the agency above it.
References:
Helsby, J., Carton, S., Joseph, K., Mahmud, A., Park, Y., Navarrete, A., Ackermann, K., Walsh, J., Haynes, L., Cody, C., Patterson, E., & Ghani, R. (2018). Early intervention systems: Predicting adverse interactions between police and the public. Criminal Justice Policy Review, 29(2), 190–209. https://doi.org/10.1177/0887403417695380
Katz, C. M., Cheon, H., Freemon, K., & Wallace, D. (2025). Evaluating the effectiveness of a police early intervention system. Police Quarterly. https://doi.org/10.1177/10986111251353487
Ridgeway, G. (2026). A conditional ordinal stereotype model to estimate police officers’ propensity to escalate force. Journal of the American Statistical Association, 1–12. https://doi.org/10.1080/01621459.2025.2597050
Walker, S., Alpert, G. P., & Kenney, D. J. (2000). Early warning systems for police: Concept, history, and issues. Police Quarterly, 3(2), 132–152. https://doi.org/10.1177/1098611100003002001
Ready to move beyond thresholds? Talk to our team about First Sign.