Member-only story
Stopped Is Not Undone
AI governance and the limits of reversing harmful automated decisions
Not a Medium member? Click here to keep reading.
Between 2012 and 27 February 2020, the Dutch Tax Administration kept a register called the Fraude Signalering Voorziening: a fraud-suspicion list that other state bodies could consult. By the time it was shut down, it listed around 270,000 people, including 1,782 who were minors when entered. The press called it a blacklist, which was unkind to blacklists. At least a blacklist should be accurate.
In 2022 the Dutch data protection authority fined the Tax Administration €3.7 million. The register had no adequate legal basis, too many employees could reach it, records were kept too long, and information was not always corrected after an investigation. An entry could help deny someone a personal payment arrangement or access to debt relief.
The fine was the familiar ending. The instructive part came afterward.
Signals had traveled beyond the Tax Administration. The state mounted what it called a final attempt to establish which organizations had received them. Its records could not reconstruct every route. In November 2022, the government could say that about 220,000 people appeared to have suffered no material adverse effect. For roughly 33,000…
