Diagnosing Prevent
A statistical evaluation of the UK’s counter-radicalisation strategy
7 June 2025
In recent years, the UK government’s Prevent programme has become a central pillar of its counter-terrorism strategy, tasked with identifying individuals deemed at risk of radicalisation before any crime has been committed. Yet for all its prominence, the question of Prevent’s actual effectiveness remains largely unexamined in rigorous, statistical terms. This essay applies a standard metric used in clinical diagnostics — the Diagnostic Odds Ratio (DOR) — to assess Prevent’s capacity to distinguish between individuals who pose a genuine threat and those who do not. Drawing on publicly available data, and making conservative assumptions where necessary, the analysis reveals that Prevent performs only marginally better than chance. It identifies very few actual terrorists, while wrongly flagging hundreds of innocent people. The evidence suggests not only that Prevent is an ineffective predictive tool, but that its design may be fundamentally flawed — raising concerns about its impact on civil liberties, public trust, and the rational allocation of security resources.
Triage
Let’s start with a basic question: why do we need to distinguish potentially useful leads from red herrings? Why can’t we just investigate every tip, regardless of how flimsy it seems? This was the mindset that predominated in the months after 9/11, when the American political scientist Ian Lustick was asked by the FBI to organise a conference bringing together social scientists and law enforcement officials. How, Lustick asked one official at the conference, did he manage to decide which tips to investigate and which to put aside as lacking sufficient credibility or importance? The response was staggering. “We chase down every tip, sir. Any one might be the next World Trade Center attack.”
No one around the table acted surprised. Lustick responded that such a policy would be utterly impossible to sustain. As a practical matter some rules would have to be devised to separate significant leads from hoaxes, errors, or inaccurate or wildly speculative suggestions. At the very least, rules for separating promising from unpromising leads would be necessary if sufficient resources would be able to be devoted to the truly important ones. But his analysis was rebuffed. The fear of being identified after another attack as the person responsible for having ignored the tip that would have prevented it was simply overwhelming. Despite the commitment of more than a third of all FBI special agents to the antiterrorism campaign, this attitude virtually paralyzed the agency’s initial response to the problem. A report from Office of the Inspector General published in June 2003 stated:
The FBI allocated massive resources to the September 11 terrorism investigation. Within 3 days of the attacks, more than 4,000 FBI special agents and 3,000 support personnel were assigned to work on the PENTTBOM [Pentagon Twin Towers Bombing] probe. Six days after the attacks, FBI Director Mueller reported that more than 500 people representing 32 federal, state, and local law enforcement agencies were working 24 hours a day at FBI Headquarters. By September 18, 2001, 1 week after the attacks, the FBI had received more than 96,000 tips or potential leads from the public, including more than 54,000 through an Internet site it established for the PENTTBOM case, 33,000 that were forwarded directly to FBI field offices across the country, and another 9,000 tips called into the FBI’s toll-free “hotline.”
The simple truth is that whenever scarce resources confront multiple demands for their use, some sort of triage is necessary. No matter how powerful the commitment was to “chase down every lead,” the FBI was forced to ignore most tips and choose only some to investigate. Looking back on the period immediately after 9/11 responsible officials told New York Times reporters that National Security Agency eavesdropping had been streaming thousands of items for follow-up to the FBI and that the “unfiltered information was swamping investigators.” Virtually all these tips “led to dead ends or innocent Americans.” Within one year the FBI had received more than 400,000 tips from the public. Of these, 85,000 were classified as “lead worthy” and sent to field offices or departments for investigation. This means that some kind of triage legal technique, whether crude or sophisticated, conscious or unconscious, resulted in the FBI effectively ignoring nearly 80 percent of the information it received from the public.
The FBI’s experience after 9/11 demonstrates that in counter-terrorism, as in medicine or emergency response, triage is not merely a matter of convenience — it is a necessity. When resources are finite and the volume of incoming information is vast, it becomes essential to distinguish the credible from the fanciful, the serious from the spurious. The core problem is thus diagnostic: how do we tell a genuine threat from a harmless individual, and do so in a way that is both efficient and reliable? This is precisely the task Prevent sets for itself. Yet if Prevent is to function as a diagnostic tool for identifying potential terrorists, then it must be assessed by the same standards we apply to any such tool. The most widely used measure of diagnostic accuracy in fields like medicine is the Diagnostic Odds Ratio (DOR) — a single figure that captures how well a test separates true cases from false alarms. Applying this metric to Prevent allows us to move beyond intuition and anecdote, and to ask the key question: how good is it, really, at telling danger from safety?
Diagnostic odds ratio
The statistic most commonly used by healthcare professionals to quantify the efficacy of a diagnostic test, taking into account both the false positive rate and the false negative rate, is the Diagnostic Odds Ratio (DOR). This measure integrates two fundamental properties of a test — its sensitivity and specificity — into a single number that captures its ability to discriminate between positive and negative cases.
A DOR of 1 indicates that the test performs no better than chance; higher values suggest stronger discriminatory power. Unlike metrics that focus solely on one type of error, the DOR offers a more balanced perspective by incorporating both types of classification failure: false positives and false negatives.
Get Dylan Evans’s stories in your inbox
Join Medium for free to get updates from this writer.
Other metrics are sometimes used in clinical contexts depending on the purpose of the evaluation. Youden’s Index, defined as sensitivity plus specificity minus one, offers a straightforward way to summarise test performance. The Area Under the Receiver Operating Characteristic Curve (AUC) reflects how well a test can distinguish between positive and negative cases across all possible thresholds, integrating the false positive rate and the true positive rate. In machine learning contexts, the F1 score is often used to balance precision and recall, but it is less common in healthcare and security settings. Among these, the DOR remains the most direct and interpretable metric for assessing overall discriminatory ability when both types of misclassification matter.
Diagnosing Prevent
To evaluate the efficacy of the UK government’s Prevent programme using this approach, we can calculate its Diagnostic Odds Ratio based on data from 2022 to 2024. Prevent is a national counter-radicalisation initiative intended to identify individuals who may be at risk of engaging in terrorism and to intervene before any crime is committed. However, its actual effectiveness in distinguishing genuine threats from harmless individuals has come under increasing scrutiny.
Four data points are required to calculate the DOR: true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN). Based on available data, the following estimates can be used: two true positives — individuals reported to Prevent who subsequently carried out an act of terrorism; 900 false positives — individuals reported to Prevent who did not go on to commit terrorism; five false negatives — individuals not reported to Prevent who nonetheless carried out terrorism; and 10,000 true negatives — individuals not reported and not involved in terrorism. This last figure is necessarily speculative, since no comprehensive public data is available on individuals who might have made statements or held beliefs falling within Prevent’s criteria but who were not reported and did not commit any violent act.
To illustrate the ambiguity of this category, one might consider recent statements by public figures such as Sir Keir Starmer, who warned that without fair immigration rules, Britain could become “an island of strangers.” Robert Jenrick, the shadow justice secretary, has argued that “excessive, uncontrolled migration threatens to cannibalise the compassion of the British public.” Both statements could fall under Prevent’s stated concerns. The government’s own refresher training course lists “cultural nationalism” as a common subcategory of “extreme Right-wing terrorist ideologies,” alongside white supremacism and ethno-nationalism. This category explicitly includes views such as prioritising integration, preserving social cohesion, and maintaining shared cultural norms — positions that, while widespread, are flagged under Prevent’s expansive criteria.
In the absence of data, I have adopted what economists call a “heroic assumption” — a simplifying leap of faith used to make a complex model workable. My estimate of 10,000 true negatives is conservative, and the real number is likely to be considerably higher.
Constructing the confusion matrix from these values, we have the following:
From this, we can calculate sensitivity (the proportion of actual terrorists correctly identified), which is 2 divided by 7, or approximately 0.2857. Specificity (the proportion of non-terrorists correctly identified) is 10,000 divided by 10,900, or approximately 0.9174.
The DOR is then calculated as:
Remember: a DOR of 1 implies the system has no discriminatory power (i.e., performs like a coin toss). A DOR > 1 implies some ability to distinguish between terrorists and non-terrorists. A DOR < 1 would imply the system is worse than random.
Not fit for purpose
This result — approximately 4.44 — means the Prevent programme has some discriminatory power, but only modestly so. A DOR of 4.44 is better than random guessing, but it remains unimpressive by clinical or statistical standards. The data reveal that for every terrorist correctly flagged by Prevent, 450 innocent people are wrongly identified. The system misses the majority of actual terrorists (5 out of 7), yielding a sensitivity of just 28.6%. This low rate of detection raises serious doubts about the utility of the programme.
What emerges is a picture of a programme that casts a very wide net, capturing large numbers of innocent individuals while failing to identify most actual threats. The potential consequences are not merely statistical. Such a high false positive rate can erode public trust, stigmatise communities, and divert attention and resources from genuine security concerns. Worse still, if the criteria for suspicion include broadly held political or cultural views — such as scepticism about immigration or calls for integration — then the programme risks pathologising mainstream opinion as latent extremism.
The DOR calculation does not merely reveal a technical shortcoming; it casts doubt on the entire conceptual framework of Prevent. If the goal is to identify future terrorists, the evidence suggests it is doing a poor job. And if, on the other hand, the real aim is to enforce ideological conformity under the guise of security, then the low discriminatory power of the programme becomes not a flaw, but a feature. Either way, the current data suggest that Prevent, far from being a precise instrument of pre-emptive safety, is a blunt and ineffective tool — one that may ultimately be doing far more harm than good. And not only that; Amnesty International has declared Prevent to be “fundamentally incompatible” with human rights. It’s high time we scrapped it.
Note: In a follow-up piece to this article, I apply cost-benefit analysis to Prevent by quantifying both the direct and indirect financial and social costs of the programme, and comparing these to the estimated benefits.
