People Analytics and Dead Salmon
How good data can still tell expensive lies.
In 2009 at Dartmouth College, when I was a first year graduate student, a team of older graduate students and a statistics professor ran an unusual experiment: they put a mature Atlantic salmon into an MRI machine designated for human brain imaging. They proceeded to present the fish with a typical cognitive task consisting of viewing different kinds of images depicting human emotion and measured blood-oxygen level dependent signal across its small brain. Upon analyzing the data, Bennett, Baird, Miller, and Wolford were able to isolate activity in specific brain areas of the salmon in response to the human emotions depicted on the screen.
There was only one minor problem: the fish was not in fact alive, and had not been alive at the time of scanning.
The experiment was used to illustrate a lesson cognitive neuroscience learned the hard way: given enough data and the wrong analytical methods, it is dangerously easy to find patterns in random noise, leading to false conclusions from ostensibly precise measurements. This lesson has stayed with me through different corporate jobs that all have had a common thread: using data to enable better decisions and moving analytics from insight to impact. And nowhere have I found the dead salmon lesson to be more consequential than in the field of people analytics.
Get Ana Bedacarratz’s stories in your inbox
Join Medium for free to get updates from this writer.
As organizations race to build workforce transformation programs, they are sadly replicating the dead salmon study more times than they realize. Armed with large datasets of engagement surveys, badge swipes, email and slack metadata, performance scores and detailed performance and 360 evaluations, they are committing one error that turns data-driven discovery into costly corporate hallucinations.
The trap illustrated in the dead salmon study was this: the more data points an analysis involves, the higher the risk of uncovering false positives: statistical patters that, like the brain activity of a dead salmon, don’t really exist. Imagine a large company analyzing years of accumulated workforce data to find a secret formula for the perfect sales rep, software engineer, or customer support representative. The more data was fed into the model, the more likely it is that the conclusion is a false positive and the company will chase a statistical ghost in whatever new hiring or automation practices they design based on this analysis.
To prevent making expensive decisions based on statistical ghosts, workforce transformation leaders should make multiple comparison correction, data splitting, and out-of-sample validation a routine part of data-driven discovery.
Until then, expecting data-driven talent strategies to improve any outcome, even in a pilot program or controlled rollout, is the same as expecting brain activity in a dead fish.
References
Bennett, C. M., Baird, A. A., Miller, M. B., & Wolford, G. L. (2012). Journal of Serendipitous and Unexpected Results. Journal of Serendipitous and Unexpected Results (jsur. org)-Vol, 1(1), 1–5.
