<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Edo van Uitert on Medium]]></title>
        <description><![CDATA[Stories by Edo van Uitert on Medium]]></description>
        <link>https://medium.com/@edovanuitert?source=rss-da969e0361ce------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Edo van Uitert on Medium</title>
            <link>https://medium.com/@edovanuitert?source=rss-da969e0361ce------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 03:30:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@edovanuitert/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why the anti-money laundering ecosystem needs an end-to-end feedback loop]]></title>
            <link>https://medium.com/@edovanuitert/why-the-anti-money-laundering-ecosystem-needs-an-end-to-end-feedback-loop-cf96351054e9?source=rss-da969e0361ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf96351054e9</guid>
            <category><![CDATA[anti-money-laundering]]></category>
            <category><![CDATA[ecosystem]]></category>
            <category><![CDATA[effectiveness]]></category>
            <category><![CDATA[publicprivate-partnership]]></category>
            <category><![CDATA[feedback-loop]]></category>
            <dc:creator><![CDATA[Edo van Uitert]]></dc:creator>
            <pubDate>Wed, 28 Oct 2026 07:31:06 GMT</pubDate>
            <atom:updated>2026-10-28T07:31:06.188Z</atom:updated>
            <content:encoded><![CDATA[<p>Financial institutions, lawmakers, supervisors and other public sector bodies invest significant time and resources in implementing and operating systems and processes designed to comply with anti-money laundering (AML) and counter-terrorist financing (CTF) regulations. At the same time, the FATF is placing increasing emphasis on effectiveness, with the fifth round of Mutual Evaluations another step in that direction. Yet despite this focus, there remains no consensus on how to assess the effectiveness of the many tools in the AML/CFT toolbox. Objective criteria are not readily available.</p><p>The Wolfsberg Group has spent several years shifting the conversation from compliance towards effectiveness. It identifies three pillars of effectiveness: 1) compliance with laws and regulations, 2) providing highly useful information to authorities, and 3) maintaining a reasonable risk-based control framework. Yet even these principles leave open the practical question of how usefulness and effectiveness should be measured in practice.</p><p>Recently, the Algemene Rekenkamer concluded that the effectiveness of anti-money laundering measures in the Netherlands is insufficiently evidenced: ‘<em>there is insufficient evidence of an effective and risk-based approach</em>’ (<a href="https://www.rekenkamer.nl/documenten/2026/03/11/gevolgen-groot-opbrengsten-onbekend---onderzoek-naar-de-anti-witwasaanpak-in-de-bankensector">link</a>).</p><p>This call for a risk-based approach is not new. For example, <a href="https://www.dnb.nl/voor-de-sector/open-boek-toezicht/thema-s/toezicht-op-financieel-economische-criminaliteit-integriteitstoezicht/qas-en-good-practices-wwft/">DNB guidance</a> from 2024 states: ‘<em>Combating financial and economic crime can be made more efficient and effective through a more risk-based approach. A more effective approach primarily means that fewer criminal funds pass through the financial infrastructure. More efficiency in fighting financial-economic crime means a reduced burden on banks and their customers.</em>’</p><p>Policy-makers share this view. In a <a href="https://open.overheid.nl/documenten/13d59d0f-652e-446b-9355-ba94b8860e46/file">letter</a> to the Dutch parliament from December 2025, the responsible minister outlined the objective of new anti-money laundering legislation. It should: <em>1) reduce the burden for legitimate citizens/companies and 2) increase the barriers for criminals.</em> These are precisely the outcomes a risk-based approach is intended to achieve.<br> <br>There is therefore broad consensus that a risk-based approach is a prerequisite for an effective AML landscape. Significant efforts has gone into implementing such an approach in the Netherlands, with support from the DNB. Why have these efforts not produced the desired results, and why has it proven difficult to demonstrate effectiveness?</p><p>My observation is that the implementation misses one crucial element: an end-to-end feedback loop.</p><p>Currently, risk-based approaches are defined and discussed at the level of individual institutions. Financial crime detection, however, is an ecosystem involving banks, the Financial Intelligence Unit (FIU), supervisors and law enforcement agencies. Without systematic feedback on which intelligence provides most investigative value, no participant can prioritize risks optimally. As a result, a truly meaningful risk-based approach requires more than institution-level optimization. <strong>It requires an effective end-to-end feedback loop and shared accountability for effectiveness across the entire chain.</strong></p><h3>The AML ecosystem</h3><p>Banks represent only one link in the reporting chain. In the Netherlands, they file Unusual Transaction Reports (UTRs) to the FIU. The FIU, in turn, makes this information available to governmental bodies such as law enforcement agencies, while also conducting its own analysis and investigations. Intelligence originating from banks and other reporting institutions is then used to identify, investigate and prosecute suspected criminals and recover illicit proceeds.</p><p>Banks, however, do not receive systematic feedback on all transactions they report. The FIU does not receive systematic feedback from law enforcement regarding the outcomes of investigations.</p><p>Although each participant is committed to performing its role well, institutions naturally only optimize for their own part of the process. No organization has the mandate to optimize the effectiveness of the chain as a whole. The paradox is that institutions are expected to make risk-based decisions, while the information needed to determine the true value of those decisions only becomes available later in the chain.</p><p>The following hypothetical examples illustrate how optimizing effectiveness within a single institution <em>could</em> unintentionally reduce effectiveness across the chain.</p><p><strong>Example 1</strong>: A bank improves one of its detection mechanisms. As a result, they report more unusual behaviour to the FIU. The FIU, however, is already dealing with a very high volume of reports. In 2025, of the approximately 560,000 subjectively reported UTRs submitted by banks, only 58,000 were marked as suspicious: just over 10% (see the 2025 <a href="https://www.fiu-nederland.nl/jaaroverzicht-2025-criminele-geldstromen-worden-slimmer-en-breder-ingezet/">FIU annual report</a>). The Algemene Rekenkamer likewise observed that the FIU in unable to process all UTRs because of the high inflow. In this context, sending additional reports to the FIU may not improve outcomes. On the contrary, it may increase the likelihood of burying truly important signals.</p><p><strong>Example 2</strong>: To improve internal effectiveness, banks may focus on detecting large transaction amounts because they appear more material. However, in areas such as cash transactions, sophisticated criminal groups may already be aware of the detection risk and deliberately distribute activity over smaller cash amounts. If that’s the case, reports on smaller cash transactions may provide more valuable intelligence than reports involving larger amounts.</p><h3>Communication within the ecosystem</h3><p>As the FATF stated in its July 2026 report on <a href="https://www.fatf-gafi.org/en/publications/Methodsandtrends/information-sharing-ppp-data-protection-arrangements.html">Information Sharing to Combat Illicit Finance</a>, effective AML/CFT systems can no longer rely on institutions acting in isolation. In a world of increasingly digital, fast-moving and cross-border financial crime, information sharing between banks, FIUs, law enforcement and other stakeholders is essential to achieving meaningful outcomes.</p><p>FATF explicitly states that effective information sharing is ‘<em>key for the effectiveness of AML/CFT systems</em>’ and highlights the urgent need for public-private collaboration in combating illicit finance.</p><p>In the Netherlands, collaboration is already taking place. Through the <a href="https://www.fiu-nederland.nl/wp-content/uploads/2023/05/FIU-Fintell-Alliance-infographic-WEB-NL.pdf">Fintell Alliance NL</a>, six Dutch banks work together with the FIU to improve the prevention and detection of money laundering/terrorist financing. The Algemene Rekenkamer specifically highlights this alliance as a positive example of collaboration. Likewise, the <a href="https://www.fec-partners.nl/">Financial Expertise Centre</a> brings together the FIU, DNB and several other governmental organizations to share information, including risk indicators and red flags.</p><p>Despite these initiatives, the end-to-end reporting chain remains fragmented. Participants continue to focus on their own responsibilities rather than tailoring their outputs to the needs of downstream partners. No-one is responsible for the overall effectiveness across the chain.</p><p>This concern is also reflected in the Algemene Rekenkamer report: ‘<em>Within the anti-money laundering chain, various parties each have their own role and access to only their respective part of the information. To identify and implement the necessary improvements to the system, it is essential that all parties have access to the relevant information.</em>’</p><p>If we cannot determine which AML interventions ultimately contribute to successful investigations, neither banks nor policymakers can make informed risk-based choices. How can we have a risk-based approach if we do not have a shared definition of effectiveness in the first place?</p><p>How could we change this?</p><h3>Learning from outcomes</h3><p>On way to demonstrate effectiveness is through a data-driven feedback loop on UTRs.</p><p>In the Netherlands, banks report the majority of subjective UTRs and account for most of the so-called suspicious declarations (the Dutch FIU determines if an UTR is actually suspicious and becomes a STR). Together with the FIU, this reporting chain could systematically indicate which reports are most valuable.</p><p>Such as system would allow banks to produce more high-value reports while reducing reports with limited or no value. This would also reduce the negative impact on legitimate citizens and businesses by reducing unnecessary interactions associated with low-value reports.</p><p>Outcome-driven learning should be embedded in every stage of the reporting chain. Doing so would transform the current system into a genuinely risk-based approach that directs attention towards real risks while reducing effort spent on less relevant activity.</p><h3>Challenges</h3><p>Although conceptually straightforward, there are several challenges that should be addressed:</p><ul><li>Privacy legislation restricts the type of information that institutions are allowed to share. Each participant has its own legal mandate and access rights.</li><li>Investigations may be compromised when information is communicated too far down the chain. Information-sharing may unintentionally reveal investigative methods and priorities, which may pose an unacceptable risk.</li><li>Feedback cycles are long. Criminal investigations can take years, meaning that improvements in effectiveness may only become visible after considerable time.</li><li>National coordination is currently lacking. Aligning public and private participants around a common objective would require significant effort and may compete with other priorities.</li></ul><p>Nevertheless, given the resources currently spent on combating financial crime by all participants of the chain, a better way to determine what works and what does not is justified. It is what is needed to make the fight against financial crime more effective whilst reducing the burden on society.</p><h3>How to start?</h3><p>Building a system that learns from investigative outcomes requires both commitment and political support. Legal blockers, where they exist, can only be addressed by law reforms.</p><p>Existing public-private partnerships could be expanded and tasked with taking ownership of the end-to-end effectiveness. They should first identify the current barriers to establishing a continuous feedback loop and learning system. Their recommendations could form the basis for law reforms to make such a system possible.</p><p>Once the legal hurdles have been tackled, implementation could start as follows:</p><ul><li>Law enforcement picks a single risk area based on national or international priorities. One example could be the <a href="https://www.fiu-nederland.nl/en/knowledge_base/cash-compensation-model/">Cash Compensation Model</a> typology. Note that the Dutch Public Prosecutor already works with thematic focus areas (see Chapter 2.2 of the <a href="https://www.om.nl/site/binaries/site-content/collections/documents/criminele-geldstromen/map/om-jaarbericht-criminele-geldstromen/om-jaarbericht-criminele-geldstromen/jaaroverzicht-criminele-geldstromen-2025.pdf">Annual Report on Criminal Financial Flow 2025</a>).</li><li>Law enforcement links relevant cases to information received from the FIU.</li><li>The FIU links those cases to transactions received from reporting institutions.</li><li>Banks link those transactions to the underlying detection mechanisms that caught that behaviour.</li><li>Banks share knowledge on the detection mechanisms that successfully caught this typology, allowing all participants to improve their methods.</li><li>Newly generated alerts on this typology are prioritized by the FIU, ensuring that law enforcement is optimally positioned to initiate criminal investigations.</li><li>Data insights are used to measure effectiveness throughout the chain, including:<br>- Number of transactions reported to the FIU relating to the typology;<br>- Number of FIU investigations based on those transactions;<br>- Number of criminal investigations initiated from FIU intelligence;<br>- Number of successful prosecutions and amount of criminal assets recovered.</li><li>Once effectiveness is demonstrated, the approach can be extended to other risk typologies.</li></ul><p>For individual risk areas, implementation may initially result in higher numbers of detected cases as institutions improve their detection capabilities. Over time, however, those numbers may decline if the typology becomes less prevalent, is successfully disrupted, or shifts elsewhere. Measuring overall effectiveness therefore requires an aggregated view over all risk typologies, rather than focusing solely on trends within a single one.</p><p>Note that prosecutions and asset recovery should not be viewed as the only measures of success. Preventive effects are harder to quantify but are equally important. Any assessment of effectiveness should therefore be based on a holistic view of real-word impact.</p><h3>Conclusion</h3><p>The Dutch AML debate has largely focused on how individual institutions should prioritize risks. Yet no institution fights financial crime alone. As long as institutions make risk-based decisions based on internal performance data, they will inevitably optimize locally rather than globally.</p><p><strong>A truly risk-based approach requires a measure of effectiveness across the chain and an AML ecosystem that learns collectively from outcomes.</strong> Only then can risk priorities be guided by real-world impact rather than institutional perspectives and activities simply driven by regulatory requirements.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf96351054e9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why measurement errors on transaction monitoring performance metrics matter]]></title>
            <link>https://medium.com/@edovanuitert/why-measurement-errors-on-transaction-monitoring-performance-metrics-matter-0d76e1e4760b?source=rss-da969e0361ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/0d76e1e4760b</guid>
            <category><![CDATA[uncertainty]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[thresholding]]></category>
            <category><![CDATA[transaction-monitoring]]></category>
            <category><![CDATA[measurement-error]]></category>
            <dc:creator><![CDATA[Edo van Uitert]]></dc:creator>
            <pubDate>Fri, 05 Jun 2026 05:46:44 GMT</pubDate>
            <atom:updated>2026-06-05T05:46:44.027Z</atom:updated>
            <content:encoded><![CDATA[<p>Transaction monitoring (TM) performance metrics are often treated as precise facts. In reality, they are estimates — subject to uncertainty. Ignoring this uncertainty can lead to suboptimal thresholds, higher investigation costs, and unnecessary client impact.</p><h3>What are measurement errors?</h3><p>Measurement errors indicate how uncertain a measurement is. The larger the error, the less certainty you have. For example, a study of global warming that concludes that global temperatures rose by 1.0 +/- 0.1 degrees between 1950 and 2025 (where 0.1 indicates the error) is far more certain of that result than another study that claims it increased by 1.0 +/- 0.9 degrees: 0.9 is a much larger range than 0.1.</p><p>When you determine performance metrics of TM systems, such as the global hit rate (the ratio of the number of true positives and the total number of alerts), you are making a measurement. If you have 5 true positives in a sample of 100 alerts, the <em>observed</em> global hit rate is 5% (= 5 / 100). But what is the measurement error? Have you not determined the hit rate exactly? Don’t we know the outcome of all alerts? Yes, the <em>observed</em> hit rate of that particular sample is exactly 5%, but a common (and often implicit) assumption is that the <em>observed</em> hit rate exactly equals the <em>true/underlying</em> hit rate of all alerts of the same type. That’s not the case.</p><p>The <em>observed</em> hit rate is an <strong>observation</strong> or <strong>estimate</strong> of the <em>true</em> hit rate. If you take another sample of 100 alerts, you might find a different number of true positives. Let’s say you find 7. This time, the <em>observed</em> hit rate is 7%, not 5%. This second sample provides a different estimate of the same underlying <em>true</em> hit rate. To reconcile these two measurements, we should provide a range of credible values (the measurement error) of the <em>true</em> hit rate, not just a single number.</p><p>But how do you determine the error on the <em>true</em> hit rate?</p><h3>Coin flips</h3><p>To help thinking about uncertainty, we start with a simple analogy: coin flips. TM alerts are similar to coin flips. Like coins, they can have two outcomes: an alert is either a false positive (the behaviour was not suspicious) or a true positive (it was suspicious) — the only difference being that the chance of a true/false positive is not exactly 50–50 as for fair coins. In most cases, the chance of getting a true positive in TM is smaller than 50%.</p><p>Let’s assume that the alerts from a particular TM system have a 10% probability of being suspicious. If this system generates 100 alerts, how many suspicious cases will we find? Similar to a coin flip experiment — where we do not always get exactly half heads and half tails — the number of suspicious cases will fluctuate and not always be 10.</p><h3>Binomial distribution</h3><p>This variability is described by a well-known statistical model: the <a href="https://en.wikipedia.org/wiki/Binomial_distribution">binomial distribution</a>.</p><p>This distribution describes how often a particular outcome occurs, for experiments with two possible results. The conditions are that the probability of each outcome is fixed and that events are independent. While both conditions hold for coin flips, TM alerts may exhibit some correlation (e.g., stemming from the same customer or rule), which can slightly affect uncertainty estimates. We will ignore this effect here.</p><p>Using the binomial distribution you can compute the probability of getting <em>exactly</em> 50 heads if you flip a coin 100 times: it is 8%. If you have patience and a lot of time, you can test this yourself. Flip a coin 100 times and record the number of heads. Repeat this experiment 10,000 times. You will find that in close to 800 (=10,000 * 0.08) experiments you ended up with exactly 50 heads.</p><p>Below, we perform a similar experiment but for TM. We take a TM system that generates samples of 100 alerts. Each alert has a 10% probability of being suspicious. We record the number of suspicious cases, which — as we saw above — can differ from 10. We repeat this process 10,000 times, resulting in 10,000 samples of 100 alerts. For each sample we record the number of suspicious cases. How does the distribution of the number of suspicious cases look? The result is shown in Figure 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*i_zpK0ajCk7jjjl7u4whdA.png" /><figcaption>Figure 1: Frequency of finding a specific number of suspicious alerts in a sample of 100 alerts with a true hit rate of 10%.</figcaption></figure><p>Figure 1 shows that in about 13% of the samples, we get exactly 10 suspicious cases. If we keep increasing the number of samples, the observed frequency approaches the exact probability. We can compute it from the binomial distribution directly: it is 13.2%. The probability of getting only 5 suspicious cases is 3.3%. This means that even in a perfectly calibrated system, a single sample can produce a much lower <em>observed</em> hit rate purely due to chance.</p><p>In practice, we can never know the <em>true</em> hit rate. We can only constrain the range of values it can take.</p><h3>Expected variation from repeated experiments</h3><p>This raises a natural question: how wide is this variation in practice?</p><p>We can determine this as follows. We order these 10,000 samples using the number of suspicious cases per sample. Then we take the central 95% of outcomes (from the 2.5th to the 97.5th percentile). In our example, this corresponds to a range of 3–15 suspicious cases. This means that if we investigate 100 alerts with a <em>true</em> hit rate of 10%, we expect the number of suspicious cases to fall in the range of [3-15] 95% of the time.</p><h3>Measurement error of the true hit rate</h3><p>In the above, we determined the expected range in the number of suspicious cases for a known <em>true</em> hit rate. In practice, we want to know the opposite: to <strong>determine the plausible range of <em>true</em> hit rates using the observed number of suspicious cases from a single sample</strong>. This sounds similar but it is a different question. Analogously to what we did above, we want to determine <strong>the 95% confidence interval: </strong>the range of plausible <em>true</em> hit rates constructed from the sample, such that if we repeat the sampling procedure many times, 95% of the constructed intervals would contain the <em>true</em> hit rate.</p><p>The binomial framework allows us to compute confidence intervals. One widely used and reliable method is the <a href="https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Wilson_score_interval">Wilson score interval</a>. If we find 10 suspicious cases in a sample of 100 alerts, this method tells us that the 95% confidence interval of the <em>true</em> hit rate is 5.5% — 17.4%.</p><p>It might surprise you how wide the uncertainty range is: from 5.5% to 17.4%. That’s far from knowing that it is exactly 10%. What can we do to reduce the uncertainty? More data! For example, if we investigate 1,000 samples and 100 of them are suspicious, the <em>observed</em> hit rate stays the same (10% in both cases) but the 95% confidence interval of the <em>true</em> hit rate shrinks to 8.3% — 12.2%. A range of 3.9% ( = 12.2% — 8.3%), compared to the 11.9% for the smaller sample: hence quite an improvement in certainty! However, investigating 1,000 alerts is more costly than investigating 100 alerts, so a balance needs to be struck between operational effort and statistical uncertainty.</p><p>In the following figure, we show 95% confidence intervals of the <em>true</em> hit rates for experiments with different sample sizes (rows) and suspicious alerts (columns).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*ge-t6kY4psLH4HKZQxwiuA.png" /><figcaption>Figure 2: 95% confidence intervals of the true hit rate based on observing a number of suspicious cases in test samples of varying sizes.</figcaption></figure><p>We can make the following observations:</p><ul><li>The confidence intervals shrink when we increase the sample size;</li><li>The width of the confidence intervals generally increases with the number of suspicious cases — the maximum width is obtained when exactly half the outcomes are suspicious. If we increase the number of suspicious cases further such that more than half the sample consists of true positives, the confidence intervals shrink again.</li></ul><h3>Accounting for uncertainty in threshold setting</h3><p>How should one take these results into account in practice? The first and foremost conclusion is this: if you investigate a sample of alerts, the hit rate you observe does not equal the <em>true</em> hit rate. Decisions should not be made on the <strong>observed</strong> hit rate, but on the confidence interval. Basing decisions on the <em>observed</em> hit rate alone can lead to incorrect thresholds, resulting in too many alerts (unnecessary negative client impact and investigation costs for the bank) or too few alerts (compliance risks).</p><p>For example, imagine we investigate a sample to determine if the threshold of a TM system is set correctly. The <em>observed</em> hit rate is 15%. What can we conclude? By itself, the number 15% is not informative. Without the confidence interval of the <em>true</em> hit rate, we don’t know if the system is performing well. We need the 95% confidence interval. If we find that the 95% confidence interval is small (e.g. 13% — 17%) — we can be pretty confident about the system’s performance. However, if it is large (e.g. 5% — 36%), we cannot conclude much.</p><p>Instead of basing threshold decisions on the <em>observed</em> hit rate, a more statistically sound approach is to base decisions on the confidence interval. A practical method would be to base decisions on the lower bound of the confidence interval. If that lower bound exceeds a certain percentage, it could be decided that the threshold should be lowered.</p><h3>Key insights:</h3><ul><li>Performance metrics of TM systems are estimates, not exact values.</li><li>The binomial distribution allows us to quantify uncertainty via confidence intervals.</li><li>Sample size strongly impacts certainty: larger samples lead to narrower intervals.</li><li>Decisions (e.g., threshold tuning) should be based on confidence intervals, not point estimates.</li><li>Ignoring uncertainty leads to avoidable suboptimal decisions.</li></ul><p><strong>Disclaimer</strong>: I’ve incorporated some textual suggestions from copilot to enhance the readability of this post. The structure and content are entirely human.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0d76e1e4760b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How much money laundering remains undetected?]]></title>
            <link>https://medium.com/@edovanuitert/how-much-money-laundering-remains-undetected-8624fe3bb3a5?source=rss-da969e0361ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/8624fe3bb3a5</guid>
            <category><![CDATA[financial-crime]]></category>
            <category><![CDATA[transaction-monitoring]]></category>
            <category><![CDATA[money-laundering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[extrapolation]]></category>
            <dc:creator><![CDATA[Edo van Uitert]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 06:12:54 GMT</pubDate>
            <atom:updated>2026-04-17T06:12:54.380Z</atom:updated>
            <content:encoded><![CDATA[<p>In this blog, we continue our discussion on how to estimate the amount of money laundered through banks. <a href="https://medium.com/@edovanuitert/how-much-money-laundering-should-a-bank-catch-5575f4c03fb8">In my previous blog</a>, we followed a <strong>top-down </strong>approach: starting from estimated national money-laundering volumes and narrowing them down to the bank level. This leads to highly uncertain — but still useful — ballpark estimates.</p><p>This blog introduces a <strong>bottom‑up</strong> method to estimate how much money laundering activity remains undetected by transaction monitoring (TM) systems. By extrapolating hit rates below alerting thresholds, banks can estimate how much suspicious activity remains undetected. This enables them to quantify catch rates — both globally and per channel — and directly assess TM effectiveness. It also allows for better‑substantiated threshold decisions. This is not just an academic exercise: these topics increasingly attract regulatory attention.</p><h3>Illustrative example for cash deposits</h3><p>We illustrate the idea with an example: a rule-based mitigant that monitors cash deposits. Through cash deposits, criminals attempt to bring dirty money into the financial system — the placement phase of the money-laundering cycle.</p><p>A hypothetical rule might generate an alert whenever a client’s cash deposit volume (sum of cash deposits) over a certain time window (e.g. one month) exceeds a threshold, say €10,000. All clients depositing more than this amount would trigger an alert and be investigated. Everything below that would remain undetected. The question is: <strong>can we estimate how much money-laundering-related cash stays below this €10,000 radar?</strong></p><h3>With mock data</h3><p>To illustrate the approach, I created dummy data. Figure 1 shows a histogram of clients grouped into bins of monthly cash-deposit volume. In this example, 110 mock clients deposited between €10,000 and €11,000 in one month.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*PHAORxt4n8Kbx2hdeErq3Q.png" /><figcaption>Figure 1: Number of clients and true positives as a function of cash-deposit volume. While most clients fall below the threshold, true positives are sparse across all bins.</figcaption></figure><p>Each bin contains an orange segment representing true positives — clients with suspicious behaviour. Because most cash deposits are legitimate, these orange parts are barely visible.</p><p>Although simplified, this example reflects two characteristics typically observed in real data: a steep decline in client counts at higher volumes and sparse true positives throughout.</p><p>The alerting threshold is shown by the vertical dashed line. Everything to the right triggers an alert, while the part on the left — greyed out — stays below the radar.</p><p>Our goal is to compare the volume of suspicious transactions that <em>is</em> detected (the orange area to the right) with the volume that is <em>missed</em> (the orange area to the left).</p><h3>Why volume matters more than counts</h3><p>At first glance, you might guess that this rule roughly captures half the money-laundering volume, since the orange areas appear similar on either side of the threshold. This is misleading, because the bins correspond to very different transaction volumes. In the first bin, suspicious volume per true positive is €500 on average (half-way between the bin edges €0 and €1,000). In the 11th bin (the first bin above the threshold), it is €10,500 — about twenty times more.</p><p>Rather than focusing on the <em>number</em> of suspicious clients, it is therefore more informative to consider the suspicious <em>transaction volume</em>. We calculate this by multiplying the number of clients in each bin by the average transaction amount for that bin (e.g. €500 for the first bin and €10,500 for the 11th). This looks as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*hbTobqaA3ZxWk6Fd6DLk9Q.png" /><figcaption>Figure 2: Alerted transaction and true-positive volume as a function of cash-deposit volume. Unlike client count, suspicious volume concentrates above the threshold.</figcaption></figure><p>The 110 clients in the €10,000-€11,000 bin together account for €1.15M in alerted transactions, of which €95K originates from nine true positives.</p><p>Most of the money laundering volume therefore lies above the alerting threshold. In this example, about 75% is captured. In practice, however, this is unknown: transactions below the threshold are not investigated and true-positive labels are unavailable. We do not know how much ‘orange’ lies below €10,000.</p><p>This is precisely what we want to estimate, since it determines which fraction of money laundering via cash deposits is captured by this rule.</p><h3>Can we extrapolate below the threshold?</h3><p>A naïve extrapolation of the orange bars in Figure 2 would extend roughly horizontally below the threshold, substantially overestimating the total money-laundering volume.</p><h3>Extrapolate using the observed hit rate</h3><p>A more reliable approach is to extrapolate the <strong>observed hit rate</strong>, defined as the ratio of true positives to total alerts. For example, in the 11th bin the observed hit rate is 9 / 110 = 8%.</p><p>The <strong>key assumption</strong> is that the <strong>hit rate generally increases with transaction volume</strong>. In the cash-deposit example, this means that we expect a higher fraction of suspicious transactions in larger cash deposit bins than in smaller ones. <strong>If this assumption does not hold, the approach breaks down</strong>.</p><p>For the cash deposits — and for many TM mitigants — this assumption is reasonable. Laundering material amounts of cash through small deposits would be inefficient. Nevertheless, it does not hold universally and should always be empirically checked.</p><p>Figure 3 shows the hit rate for the mock data. Orange dots represent the observed hit rates per bin (below-threshold values are shown only for illustration). The grey dashed line is the true underlying hit-rate trend used to create the mock true positives. Sampling noise explains why the orange points deviate from this line. Issues of noise and statistical uncertainty are discussed in more detail in a forthcoming blog post.</p><p>To extrapolate below the threshold, we fit a function to the observed hit rates above the threshold and apply it below. Here, we fit a second-order polynomial constrained to be positive and to pass through the origin. The fitted curve is shown by the blue dotted line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*1-97C-GrWHS1k5TYHb-fxg.png" /><figcaption>Figure 3: Hit rate as a function of cash deposit volume. By fitting a function to the observed hit rate above the threshold, we can extrapolate the hit rate below the threshold.</figcaption></figure><p>Next, we multiply the extrapolated hit rate below the threshold by the known transaction volume per bin. This provides an estimate of money-laundering volume via cash deposits below the threshold. Comparing this with the observed suspicious volume above the threshold gives an estimated catch rate, i.e. the percentage of all suspicious cash deposits captured by this rule. In this example, the estimated catch rate is 77%, close to the true value of 75%.</p><p>This approach also allows for estimation of suspicious volume just below the threshold, e.g. in the [€9,000 - €10,000] bin. This indicates how much extra suspicious activity would be captured if the threshold were lowered, and can be weighed against the resulting increase in false positives — useful input for threshold substantiation.</p><p>Extrapolation becomes less reliable further from the threshold, but this is partly offset by the decreasing average transaction volumes in the lower bins.</p><h3>Does this hold for real data?</h3><p>This is a highly idealized example. The same functional form used to generate the mock true positives was used for fitting, and we made sure it resulted in enough true positives to fit the data.</p><p>Real data present additional challenges. Different functional forms should be tested and compared to assess bias and stability. Uncertainty in the observed hit rate — ignored here — should be incorporated in the fit, and confidence intervals should be derived for the estimated missed suspicious volume.</p><p>In short, assumptions must be explicit and testable, and uncertainty must be quantified. This is precisely a type of problem data scientists are trained to address. A rigorous implementation makes this method suitable for use in TM effectiveness assessments and internal model-validation discussions.</p><h3>Strengthening the extrapolation</h3><p>The reliability of the extrapolation can be improved through below-the-line sampling: investigating a sample of clients from a bin below the alerting threshold to generate extra data points for the fit.</p><p>But remember: if hit rates do not increase monotonically with risk intensity, or if too few true positives exist above the threshold to estimate a stable trend, the extrapolation becomes unreliable and should not be used.</p><h3>Estimating total money-laundering volume</h3><p>Money laundering happens through different channels. Banks employ multiple mitigants to capture different aspects of money laundering risks — also for cash withdrawals, transactions to and from high risk countries, etc. Alerts from different mitigants may overlap, and thresholds may vary across client segments. In some segments, suspicious activity is rare.</p><p>Combining per‑mitigant extrapolations while accounting for overlap quickly becomes intractable.</p><p>A more feasible alternative is <strong>to fit a</strong> <strong>supervised machine-learning</strong> model on historical suspicious cases. Such a model captures patterns from the data across risk types, transaction categories and client segments. By extrapolating hit rates across bins of model scores — analogous to the cash-deposit example — we can estimate suspicious volumes even in regions with limited historical alert coverage. Below-the-line samples could again be used to improve the reliability of the extrapolation.</p><p>Conceptually, this is the same extrapolation problem as before — but instead of a single rule and threshold, the model learns across all dimensions simultaneously. Extrapolating one model would suffice to estimate suspicious transaction volume across the entire client portfolio.</p><h3>How can these results be used?</h3><p>Although estimates of missed suspicious volume will be imperfect, they are still useful for:</p><ul><li><strong>Estimating overall catch rates </strong>as part of TM effectiveness assessments.</li><li><strong>Performing gap analyses </strong>by comparing bottom-up and top-down estimates.</li><li><strong>Estimating channel-specific risk contributions,</strong> such as for cash deposits, which the top-down approach cannot provide.</li><li><strong>Threshold optimization </strong>for individual rules/models, balancing the captured suspicious volume with the number of false positives.</li></ul><h3>Key insights</h3><ul><li>A bottom-up approach can estimate the volume of suspicious transactions that remain undetected.</li><li>The method extrapolates the observed hit rate below the alerting threshold to estimate missed suspicious activity.</li><li>Applied globally, it provides insight into the overall catch rate — an important measure of TM effectiveness.</li><li>Applied to individual mitigants, it supports threshold optimization and channel-level analysis.</li><li>A robust application requires making explicit assumptions, uncertainty quantifications and below-the-line samples. While uncertainty can never be eliminated, making it explicit and quantifiable is a crucial step towards more transparent, defensible, and effective transaction monitoring.</li></ul><p><strong>Next blog</strong>: in my next blog I plan to dive into estimating uncertainties of observed hit rates. How can we estimate them, but above all, why should we care?</p><p><strong>Disclaimer</strong>: I’ve incorporated some textual suggestions from copilot to enhance the readability of this post. The structure and content are entirely human</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8624fe3bb3a5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How much money laundering should a bank catch?]]></title>
            <link>https://medium.com/@edovanuitert/how-much-money-laundering-should-a-bank-catch-5575f4c03fb8?source=rss-da969e0361ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/5575f4c03fb8</guid>
            <category><![CDATA[banks]]></category>
            <category><![CDATA[financial-crime]]></category>
            <category><![CDATA[transaction-monitoring]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[money-laundering]]></category>
            <dc:creator><![CDATA[Edo van Uitert]]></dc:creator>
            <pubDate>Wed, 18 Mar 2026 07:38:34 GMT</pubDate>
            <atom:updated>2026-03-18T07:38:34.252Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Why estimating the “catch rate” is hard — but still matters.</em></p><p>Automated transaction monitoring (TM) systems are used by large financial institutions (FIs) to detect unusual transactions that may indicate money laundering or terrorist financing. In this blog series, I share experiences and personal opinions from developing machine learning/AI models in this domain.</p><p>In my <a href="https://medium.com/@edovanuitert/how-do-you-assess-the-performance-of-a-transaction-monitoring-system-786e62b3c696">previous blog</a>, we dove into the hit rate, i.e. the proportion of TM alerts that turn out to be true positives, and how it can be used to calibrate alerting thresholds. It’s an important metric to consider. However, it doesn’t tell us <strong>how much of the total money laundering volume flowing through a financial institution is captured</strong>. That’s described by the catch rate.</p><h3>The catch rate</h3><p>The <strong>catch rate</strong> (technically: recall) is defined as:</p><p>Catch rate = (Number of true positives caught) / (Number of true positives occurring within a financial institution),</p><p>where a <em>true positive</em> refers to unusual transaction behaviour that might relate to financial crime.</p><p>So if there are 100 instances of suspected money laundering and the system detects 10, the catch rate would be 10%.</p><p>It is good to note that FIs report transactions that <em>might be</em> related to money laundering and terrorist financing; the fraction that <em>is actually related</em> is unknown to them. We will come back to that later.</p><p>Banks are expected to catch a reasonable portion of all money laundering. If criminals launder €5 billion per year through a bank but only €50 million would end up in unusual/suspicious transaction reports, something would be off. The challenge for banks is that they don’t know how much money is laundered through their systems. Criminals don’t label their transactions.</p><p>If banks had unlimited resources, they could investigate all customers and get a complete picture. Unfortunately, this is practically impossible for most banks as they have far too many customers.</p><p>What options do banks have to estimate their catch rate? Broadly speaking, two paths exist:</p><ul><li>a top-down approach (start from national money laundering volumes and narrow it down);</li><li>a bottom-up approach (estimate the missed false negatives using extrapolation and sampling of deployed rules/models).</li></ul><p>Here, I’ll describe the top-down approach. The bottom-up one I will explore in my next blog.</p><h3>Top-down approach: the Drake equation</h3><p>If you have a background in astronomy (like I do), you’ve probably heard of the <a href="https://en.wikipedia.org/wiki/Drake_equation">Drake equation</a>. It’s a famous back-of-the-envelope formula that estimates the expected number of extraterrestrial civilizations in our Galaxy. It multiplies several uncertain factors, each describing a narrowing aspect (number of stars in the Galaxy, fraction of stars with planets, number of planets per star, fraction of planets that might be habitable, etc.).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/757/1*Lq5tIJ1UFE539qd8n0tzRA.png" /></figure><p>The equation might surprise you for its simplicity, but that is deceptive. Some of these factors are <em>really</em> hard to estimate and remain highly uncertain. Combining all the uncertainties on the different factors, the resulting uncertainty on the expected number of extraterrestrial civilizations is huge.</p><h3>The Drake equation for money laundering</h3><p>In the spirit of the Drake equation, we can write down an equation for the expected volume of unusual or suspicious transactions that a bank should report. By comparing that to the volume that is actually reported, we can estimate the catch rate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sA7TR3aaETkzLT00wsrkSQ.png" /></figure><p>Below we explain each factor.</p><p><strong>Starting point: Total volume of money laundering in a country</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0oSnrHAbyD56twoXZrRaLg.png" /></figure><p>Academic estimates exist for some countries. For example, the money laundering volume in The Netherlands in 2014 was estimated to be €13 ± 2 billion (<a href="https://repository.wodc.nl/bitstream/handle/20.500.12832/2319/2790_Volledige_Tekst_tcm28-355586.pdf?sequence=2&amp;isAllowed=y">Unger et al.</a>). Adjusting this for inflation, this becomes roughly €18 ± 3 billion today.</p><p><strong>How much of this flows through banks?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CJ0jwGkZHg2-oFUqsfQebg.png" /></figure><p>Not all money laundering flows through banks; some goes through Payment Service Providers, cryptocurrency platforms, gambling providers, underground banking, etc.</p><p>We can estimate this factor using the 2024 <a href="https://www.fiu-nederland.nl/wp-content/uploads/2025/07/FIU-annual-review-2024.pdf">report</a> of the Dutch Financial Intelligence Unit (FIU-NL), the government body to which all reporting organizations send their unusual transactions. Table 8 in that report shows that 55% of suspicious transactions (i.e., transactions that are linked by the FIU to a criminal investigation) originate from banks, though this is a crude proxy with strong assumptions.</p><p><strong>How much flows through a specific bank?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZRO-iwkPkx9b6PW3xDYrbQ.png" /></figure><p>The market share can be estimated from the relative size of the balance sheet compared to the entire sector. A bank with a 20% market share is expected to handle roughly 20% of transactions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1012/1*tZA3VwjnhcvChVcEopGl6Q.png" /></figure><p>Banks differ in client types, products, and controls. Banks that service public institutions are likely less exposed to money laundering compared to e.g. a fast-growing digital bank. A plausible range for this factor is 0–2, but the true value cannot be estimated by individual banks as they cannot compare their risk profiles with other banks.</p><p><strong>How much money laundering is detectable?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*ZeAb_5F7jbraa4bhqFqzUA.png" /></figure><p>Even if analysts investigated all clients, some money laundering patterns would remain indistinguishable from normal behaviour and therefore evade detection. The range for this factor is 0–1.</p><p><strong>How well does the bank’s TM system perform:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/957/1*cAXRFuBZsZiTkBHqCog_DQ.png" /></figure><p>No bank is able to catch 100% of all unusual and potentially suspicious behaviour. A reasonable range might be 0.5–1.0, depending on the bank’s risk appetite (the amount of risk they’re willing to accept) and maturity.</p><p><strong>Of all detected unusual transactions, how many are related to money laundering?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*vod9Lz9JiN7VjUKtuv1LeQ.png" /></figure><p>This is one of the biggest sources of uncertainty. Banks have no insights into how many of their unusual transactions are linked to money laundering.</p><p>Using the same FIU-NL 2024 report, we can derive a crude estimate, accounting for the number of unusual transactions and confirmed suspicious transactions, and for the relative frequency of money laundering as a registered form of crime (their Table 9). This implies a low conversion rate (roughly 1 to 13), but that likely is a significant underestimate because:</p><ul><li>The FIU does not have the capacity to investigate all reported unusual transactions;</li><li>More unusual transactions may be linked to criminal investigations in the future.</li></ul><p>Hence what is a reasonable value for this conversion factor? 4? Or 8? It’s highly uncertain. The FIU might be able to provide an estimate by analyzing a random sample of each FI’s unusual transactions.</p><p><strong>How often is the same money laundering flow reported more than once?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*Q591SYh-YrFkKUOlXfzCSA.png" /></figure><p>Money laundering consists of different phases: <a href="https://en.wikipedia.org/wiki/Money_laundering">placement, layering and integration</a>. Different banks may report different stages of the same flow.</p><p>For example, imagine a criminal depositing a large volume of cash at bank A (placement). Bank A detects this and reports it to the FIU. The criminal might transfer their funds to bank B. From that account, they purchase a large volume of cryptocurrency (layering). Bank B detects this and files it to the FIU. The same money laundering flow would be reported twice.</p><p>How often this happens is unknown, but a credible range for this factor is 1- 2.</p><h3>The outcome: ballpark estimates with huge uncertainties</h3><p>Since the top-down approach is based on several highly uncertain factors, the resulting estimate becomes very imprecise. Even if we knew each factor to within 20% (which, for most factors, is way too optimistic), the resulting uncertainty would be √8 × 20% ≈ 57% (standard error propagation).</p><p>So if our best guess for the expected volume of unusual transactions for a bank would be €5 billion, the uncertainty range of this volume translates into €2.2 — €7.9 billion.</p><p>It’s a useful ballpark estimate, but certainly not accurate enough to benchmark performance reliably. However, if a bank only reports e.g. 5% of this ballpark estimate, it might be a sign that important risks are overlooked. However, if they report 10x more than expected, they might be over-reporting, which could lead to excessive and avoidable negative client impact and operational costs.</p><p>Note that Frank Drake did not intend his equation to be actually used to quantify the number of extraterrestrial civilizations, only to stimulate discussion. Unsurprisingly, perhaps, it was still used. The best estimate it provides for the number of civilizations ranges from 1 (if you count us) to 50 million. Our uncertainty is luckily not that large.</p><h3>Alternative angle for a top-down approach</h3><p>The FIU has insight into the full landscape of unusual and suspiciously-marked transactions across all institutions. If they shared anonymized ratios on the conversion rates of unusual suspicious transactions per bank, adjusted for each bank’s market share, banks would gain useful insights into their relative catch rates.</p><p>Causes of differences in relative catch rates would remain ambiguous, though. For example, imagine two banks with a similar client portfolio and risk profile. If the unusual transactions of bank A become twice more often suspicious, this might imply that bank B is missing money laundering risks. However, it could also mean that bank A has weaker controls for onboarding new clients, with more criminals in their books as a result. Regardless of the cause, this would provide useful complementary insights to each bank’s internal assessments.</p><h3>Key insights</h3><ul><li>Another important performance metric of TM systems is the catch rate. This is defined as the proportion of potential money laundering volume a bank reports compared to the total volume of money laundering that is taking place.</li><li>The catch rate is generally unknown but can be estimated.</li><li>One way of estimating the catch rate is a top-down approach, starting with the volume of money laundered nationally and narrowing it down through several uncertain factors, deriving an estimate of the volume of unusual transactions per bank.</li><li>These uncertainties accumulate, making the final estimate very imprecise. It might still provide a useful ballpark estimate, however.</li><li>The FIU could provide valuable complementary insights on the relative catch rates of reporting institutions.</li></ul><p><strong>Next blog</strong>: in my next blog I plan to dive into the bottom-up approach as an alternative way to estimating the catch rate.</p><p><strong>Disclaimer</strong>: I’ve used copilot for light textual editing to enhance the readability of this post. The structure and content are entirely human.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5575f4c03fb8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How do you assess the performance of a transaction monitoring system?]]></title>
            <link>https://medium.com/@edovanuitert/how-do-you-assess-the-performance-of-a-transaction-monitoring-system-786e62b3c696?source=rss-da969e0361ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/786e62b3c696</guid>
            <category><![CDATA[alerting]]></category>
            <category><![CDATA[financial-crime]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[anti-money-laundering]]></category>
            <category><![CDATA[transaction-monitoring]]></category>
            <dc:creator><![CDATA[Edo van Uitert]]></dc:creator>
            <pubDate>Fri, 20 Feb 2026 07:22:21 GMT</pubDate>
            <atom:updated>2026-02-20T07:22:21.403Z</atom:updated>
            <content:encoded><![CDATA[<p>Detecting and reporting unusual transactions that could be related to money laundering or terrorist financing is a core responsibility of banks and other financial institutions. Due to the high volume of transactions, large organizations rely on automated transaction monitoring (TM) systems. These systems output alerts that are investigated by human analysts who assess whether it’s truly suspicious and should be reported to the authorities (a true positive) or poses no risk (a false positive).</p><p>Over the past years, my colleagues and I have been developing machine learning models to improve TM systems by a) capturing more transaction behaviour that is suspicious, b) generating less false positives and c) do so in a robust/transparent/ethical way.</p><p>In this first post of what I hope will become a series of short articles/blogs, I would like to share some insights and personal opinions. This will hopefully benefit other AML (Anti-Money Laundering) professionals of various backgrounds — data scientists, rule developers, Compliance experts — and perhaps also offer curious outsiders a look behind the scene.</p><p>In my first article, I’ll focus on a central question: how to measure the performance of a transaction monitoring system.</p><p>My main objective is to show that <strong>the default metric used in the industry, i.e. the global hit rate or true positive ratio, misses</strong> <strong>important insights into how your TM system is functioning</strong>, potentially leading to suboptimal decisions regarding alerting thresholds and redevelopment opportunities. As we’ll see, a more informative metric is the <strong><em>local hit rate</em></strong><em> at the alert threshold</em> — the probability that <em>the next</em> alert at the threshold boundary is a true positive.</p><h3>Defining the global hit rate</h3><p>Let’s start with formally defining the global hit rate. It is computed by counting the number of TM alerts that are considered true positives — i.e. those that are interesting/investigation-worthy/potentially suspicious — divided by the total number of TM alerts:</p><p>Global hit rate = #True positives / #Total alerts</p><p>In plain language: the global hit rate measures the proportion of alerts that turn out to be useful. It does <em>not</em> measure how well the system captures suspicious activity overall. The latter is challenging to measure. I plan to dive into that in my next blog.</p><p>The inverse metric of the hit rate is also frequently used: the (global) false positive ratio, which is the number of false positive alerts divided by the total number of alerts:</p><p>Global false positive ratio = #False positives / #Total alerts = 1 — Hit rate</p><p>Most transaction monitoring systems are reported to have global false positive ratios in the range 90–95%, with corresponding global hit rates of 5–10% [e.g. see <a href="https://www.mckinsey.de/~/media/McKinsey/Business%20Functions/Risk/Our%20Insights/The%20new%20frontier%20in%20anti%20money%20laundering/The-new-frontier-in-anti-money-laundering.pdf#:~:text=Source%3A%20McKinsey%20analysis%2090%20percent,many%20banks%20and%20then%20design">this McKinsey study</a> or <a href="https://www.sciencedirect.com/science/article/pii/S0167739X24002607">this paper</a>]. This means most alerts are not related to financial crime. This is not surprising: distinguishing normal from suspicious behaviour is technically challenging, because criminals intentionally avoid patterns that are easy to detect.</p><h3>A simple example: cash deposits</h3><p>Consider the case of cash deposits. Cash deposits are a known way for criminals to place criminal proceeds into the financial system. Hence banks generally have TM systems in place that monitor unusual cash deposits. A simple rule might generate an alert when a client’s cash deposit volume (sum of transactions) exceeds a threshold, say €10,000. Again, in practice we expect such a rule to generate many false positives as there are simply a lot of customers who legitimately use cash.</p><p>Now imagine that this cash deposit rule turns out to have a global hit rate of 15%, so only 85% of the alerts are false positives. Well done! Not only would this be considered a pretty neat performance, but people might actually start to worry: <em>are we not missing risk below the threshold</em>?</p><p>The standard approach to assess this is via below-the-line testing. This entails virtually lowering the alerting threshold (e.g. from €10.000 to €9.000) and investigating the additional alerts. If those contain enough true positives, you might want to lower the threshold in production.</p><p>So, should we initiate a below-the-line test then? By looking at the global hit rate or false positive ratio alone, <em>you cannot tell whether that’s a good idea</em>.</p><h3>Why the global hit rate fails: two TM systems</h3><p>To illustrate this, we look at how the hit rate might vary as a function of cash deposit volume. In general, we expect that higher deposit volumes are more likely to be suspicious. Small cash deposits would simply be a very inefficient way of laundering large volumes of criminal proceeds. Hence if that trend is not visible, the TM system may not adequately capture the risk. I’d even like to postulate that if the trend is not generally increasing, you should consider redeveloping the TM system as you’ve no way of telling whether it sufficiently captures the risk (topic of my next blog).</p><p>Imagine we have set up two alternative systems for capturing cash deposits, which have a different technical implementation. For example, one system may consider the sum of cash deposits over the past week, while the other measures it over a longer time window. As a result, the relationship between deposit volume and hit rate will be different.</p><p>We visualize two potential outcomes below. Both have the same global hit rate of 15%. However, the shape of the relationship differs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*97jV4YaSIGyvdptE1jTfQQ.png" /></figure><p><strong><em>TM System A</em>: has a high discriminatory power</strong></p><ul><li>The hit rate increases sharply towards higher volumes.</li><li>The suspicious behaviour is concentrated in a narrow volume range.</li><li>At the alerting threshold, the local hit rate is already pretty low (3%).</li></ul><p><strong><em>TM System B</em>: has a low discriminatory power</strong></p><ul><li>Has the same global hit rate as <em>TM system A</em>.</li><li>The hit rate stays around ~15% across a wide range of volumes.</li><li>At the threshold, the hit rate is still relatively high (12%).</li></ul><p>Would it make sense for these TM systems to adjust the alerting threshold? Should we start with a below-the-line sampling test to make sure?</p><p>For <em>TM system A</em>, the answer is no. The local hit rate at the alerting threshold is already low (3%). Lowering the threshold would almost exclusively add false positives (less than 3% would be true positives).</p><p>For <em>TM system B</em>, the local hit rate is 12%. Lowering the threshold would make more sense as the additional alerts contain over 10% of true positives, i.e. there is still quite some risk left below the threshold.</p><p>If anything, it would make sense for <em>TM system A</em> to <em>increase</em> the alerting threshold (e.g. aiming for a local hit rate of 4% or 5% or so — the actual value should align with the risk appetite of the financial institution). This would reduce a lot of false positives and corresponding negative client impact, whilst not missing too many true positives (although that also depends on the shape of the alert distribution). As a result, the global hit rate might increase to 20–25%. The investigative capacity that is saved this way could be redirected at other TM systems that have a better performance.</p><p>This example demonstrates the core insight: <strong>the</strong> <strong>global hit rate alone does not tell you whether your threshold is set correctly</strong>.</p><h3>Which TM system to improve</h3><p>Now let’s assume TM system A and B are addressing two different risks. The bank needs to use them both. If you have time to redevelop one system, which one would you pick?</p><p>Based on the global hit rate (15% in both cases), you wouldn’t be able to tell. But the local hit rate shows that <em>TM system A</em> in much better in concentrating the risk. Concentrating the risk even further gets increasingly difficult. On the other hand, <em>TM system B</em> has more room for improvement, and would be the more logical choice for redevelopment. Of course, reality is more nuanced and there are additional elements to consider, but at least this offers a clear data-driven recommendation.</p><h3>Key takeaways</h3><ul><li><strong>Always visualize the relationship</strong> between the quantity/metric you alert on (e.g. cash deposit volume) and hit rate. It will show you whether your threshold is set appropriately.</li><li><strong>Do not rely on the global hit rate alone</strong>. It does not tell you whether the threshold needs to be adjusted, nor whether the TM system could be further improved.</li><li><strong>Use the local hit rate</strong> — the hit rate at the threshold — to guide threshold-setting discussions.</li><li><strong>Aim higher as an industry</strong>: Global false positive ratios of 90–95% should not be considered sufficient. Even 85% leaves substantial unnecessary work and negative client impact. In this age of endless AI possibilities, let’s be more optimistic and aim for false positives ratios of 50% in 2030 — quite a modest objective compared to the AGI claims of the global tech leads, no?</li></ul><p>Now I realize that the above may seem obvious (if it does, that’s good!). However, if you’re in a field for a while, you might become used to certain perspectives such as the global hit rate, which can blind you to their limitations. As this post shows, you may miss out on the full story — and getting a more complete view doesn’t cost you anything.</p><p><strong>Next blog</strong>: in my next blog I plan to dive into the topic how to assess whether a particular money laundering risk is sufficiently captured by a TM system. Is there a way to measure this? Stay tuned…</p><p><strong>Disclaimer</strong>: I’ve used copilot for light textual editing to enhance the readability of this post. The structure and content are entirely human.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=786e62b3c696" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>