Sitemap
Anouk Ruhaak

Anouk Ruhaak

Data governance.

Data Overload

9 min readJul 16, 2024

--

Press enter or click to view image in full size
Photo by Evan Demicoli on Unsplash

Why do we collect data? It seems like an innocent enough question, but in my experience the answer tends to be as quick as it is misguided. And many answers are simply wrong.

Plenty has been written about how more data does not necessarily lead to better decisions. Adding more data can merely create more noise, resulting in information overload or decision fatigue, not more insights. It’s tantamount to asking for input on a forum. Once you have accumulated a couple of perspectives, you really do not need another 89 commenters to chime in. All that does is create more trees through which to miss the forest.

Wasteful data collection becomes even more problematic when the data in question is of a personal or otherwise sensitive nature. Whenever we collect data, or whenever data about us is collected by others, we risk that data falling into the wrong hands or for it to be used against our interests. For this reason, most data protection laws mandate data minimization: we can only collect personal data that is needed to achieve a clearly communicated purpose, and nothing more.

And yet, many of the projects I have worked on over the years started with the premise that if only there were more data, things would be some version of better. What ‘better’ would look like is rarely well defined. So why does this keep happening? Why are we wasting precious resources — not to mention valuable fuel — on data collection efforts? Here are some of the fallacies I have found along the way and how to avoid them.

Procrastination Data

To change nothing, but to be seen as doing something — analysis paralysis is a powerful form of procrastination. And so companies and governments alike end up spending millions if not billions to erect large data infrastructures so as to collect more data about more things. Money is being spent, people are expending energy, things are happening! Except maybe they’re not. And perhaps, while we’re spending our resources on collecting data, we forget to actually focus on doing the very things that would make a difference.

The problem with procrastination data is that there is an opportunity cost to collecting data. While we are focusing on building a piece of infrastructure, we may be ignoring the very thing we set out to solve. And worse still, at times the data collection effort itself becomes a reason to hold off on doing anything else. This happens when we hold off on making urgent decisions until the data is finally collected — even if in reality we have enough information to make that decision today.

This is one of the hardest fallacies to avoid, as those that collect the data often benefit from the fact that they are seen as doing something, while those that suffer the consequences may not have a voice. For instance, during the pandemic a lot of governments spent significant resources on collecting and organizing COVID related data. In some countries, this data informed strategic decisions about social distancing, lockdown or the allocation of scarce resources such as masks. However, in many cases the data was collected but never acted on. Those who suffered from this inaction — for instance vulnerable populations — rarely had a seat at the table.

Data without a cause

Can’t decide what problem you want to solve? Can’t agree on a specific problem definition? Instead of doing the hard work of actually agreeing on a shared problem, or a clear purpose, it’s often far easier to just start hoovering up data. Unfortunately, when you do not know how you will use this data or what you will use it for, it becomes quite hard to ensure you’re collecting data that will actually be useful down the road. We often see this fallacy emerge in data collection efforts that include many different stakeholders. These include smart city projects or global health initiatives, where governments, companies and service providers all need to collaborate. Instead of making sure everyone is aligned on a core purpose, such an effort can suffer scope creep when it tries to appease everyone. Before long, the original intent of the endeavour is lost in the weeds.

The solution here is far easier said than done: agree on a shared purpose that is specific enough to be meaningful. For instance: ‘we collect data to improve our meetings’ is very vague, whereas ‘we collect data about the length of meetings, speaking time and participant satisfaction to inform how we organize meetings in the future’ allows us to make precise decisions about what data to collect. The benefit of having a clear purpose is that it allows us to make hard trade offs down the road.

FOMO data

Sometimes we collect data because we fear that we may miss out if we don’t. I would speculate that a lot of web and IoT analytics data are collected for this very ‘reason’. We may not know why we need detailed information on who visits our website, but it might be useful in the future. The arrival of Large Language Models (LLMs) has made this fallacy even more prevalent, as virtually any kind of data might one day be useful to train a nascent AI.

The problem here is that data collected for absolutely no other purpose than ‘we may need it one day’ tends to not be very well organized, meaning that it is likely not going to be very useful down the road. And of course, this kind of data collection violates most data protection laws.

Get Anouk Ruhaak’s stories in your inbox

Join Medium for free to get updates from this writer.

So how do we resolve this? Before you start collecting data for no reason, ask yourself what future use cases you expect to have for this data, how many resources you would be willing to commit to ensuring that this data is properly protected, maintained and structured (in accordance with that future use case) and, critically, whether it’s legal to collect this data. If you do not have clear answers for any of these, or if the answer is ‘no’, then take a deep breath and go do something else.

Mind bending data

Here we find the illusion that if we just had better facts we might change someone’s mind. Facts don’t change minds. To change minds you’ll likely need a persuasive, if not emotional, argument. We know this, but somehow knowing this fact has yet to change our mind about it. It’s so tempting to think that someone, or a group of people, would make better choices if only they knew and could see the impact their current choices were having on their well-being. If only we could provide smokers with more evidence that smoking will lead to cancer, surely they would quit. The reality is that behavioural change is far more complex than the simple insertion of more information.

How do we avoid this? Instead of collecting data with the intent to change someone’s mind, collect data to understand why they are committed to their current stance. What is this opinion or behaviour doing for them? Once we’re able to answer these questions we are far better positioned to actually encourage change.

And of course there are many instances where data does inform behavior. For instance, we may adjust our driving speed upon receiving information from our dashboard that we’re exceeding the legal speed limit. The crux here is that in this instance we are already committed to driving below the speed limit (or to avoiding fines).

Data for control

A lot of data collection efforts focus on monitoring project outcomes or reviewing the impact of allocating resources towards a specific outcome. This is especially true in situations where those providing the resources (e.g. a donor or funder) are not the same actors as those implementing the project (e.g. the recipient of funds). In other words, monitoring becomes especially salient in situations where there is an information asymmetry.

While the impulse to want to ensure money is not being wasted is very understandable, the need for control and monitoring can itself create inefficiencies. The burden of reporting can become so cumbersome as to undermine the initial purpose of the funding arrangement.

How do we avoid this? For one, we need to be careful to not let one bad actor ruin things for everyone else. Occasionally a bad actor will run off with your money or engage in other corrupt practices. Naturally, when something like this happens we want to make sure it never ever happens again. However, when the costs of preventing corruption start to outweigh the costs of the actual corruption — especially when well-meaning actors are now forced to implement additional reporting mechanisms — it’s time to reconsider our priorities. Just because you can prevent bad things from happening, doesn’t always mean it’s the most effective course of action.

Secondly, we should optimize for collecting data that actually encourages a feedback loop and can meaningfully inform decisions about next iterations of the funded project. That is, we should collect data that can help recipients improve their program designs and delivery, rather than merely providing a funder with a report.

Precision data

Ever since data collection became easy and relatively inexpensive, we’ve collectively developed a bit of an obsession with precision: from precision agriculture to precision medicine, if it can be measured it will be collected. While precise data can be tremendously useful, context is key: in some contexts knowing the exact measurements can save lives (e.g. space flight), while in others the time and energy spent on more precision add little if any additional value.

Nowhere is a misguided obsession with precision more pronounced than in advertising. Billions are spent collecting fine-grained information about our online behaviours; information that is subsequently turned into personal profiles that are used to bombard us with hyper-targeted ads.

The flaws in precision advertising are threefold. One, just because something is precise doesn’t mean it’s right. Either because the data is simply wrong — for instance when Facebook was convinced I am an 18 year old because I gave it the wrong birthdate. Or, because the data doesn’t mean what you think it does: I may be 18 (I’m not), but that does not mean that I like tank tops, even if other 18 year olds do like those things. Relatedly, precise observations often merely represent one point in time that quickly becomes outdated. I may have spent an hour looking for a blue miniskirt with yellow buttons, but this does not mean that I will forever be interested in such an item, or even in skirts in general.

Thirdly, just because something is precise does not mean that knowing it will make you more effective. Take the example of Ster, a company responsible for the broadcast of radio, television and online ads on Dutch public media channels (the Dutch version of PBC). Two and a half years ago, Ster decided to switch from targeted advertising based on cookie data, to contextual advertising where the ads you see are solely based on the content of the page you’re visiting. For instance, if you’re visiting the website for a cooking show you might receive recipe ads. Ster found that this type of advertising was at least as effective as its targeted counterpart, while spending less money. Given the evident success of this alternative, why is hyper-targeting still a thing?

The uncomfortable truth is that the myth of precision is not just kept alive by the platforms that are selling access to users and advertising space. It’s also reinforced by the very critics who condemn those platforms for providing the tools for mass manipulation. Their critique very much hangs on the notion that hyper-targeting is effective and should thus be banned. The best way out of this fallacy is therefore to take a step back and consider what you hope to achieve and to review the possible strategies and their effectiveness, instead of sticking with the default path.

I hope the above has made clear that there are many tempting reasons to collect data. But while collecting data may be trivially easy, collecting the right data in the right way requires us to carefully think through what it is we hope to achieve, what kinds of data would actually help us achieve those goals, as well as a careful consideration of the opportunity costs of collecting data.

--

--