Sitemap
Marc Bolle

Marc Bolle

📊 Data consultant exploring Tech 🚀🌟

Datasaurus Dozen Visualization using Python

5 min readNov 18, 2023

--

Press enter or click to view image in full size
Image by Cup of Couple on Pexels

When conducting data analysis, relying solely on summary statistics such as the mean, variance, and correlation might result in missing import insights. In this post, I’ll demonstrate the importance of data visualization by introducing the Datasaurus Dozen.

What is the Datasaurus Dozen?

The Datasaurus was initially introduced by Professor Alberto Cairo in a blog post in 2016. It’s a usual dataset composed of 142 observations with a bivariate normal distribution. Yet, when visualized on a scatterplot, these observations collectively form the shape of a dinosaur:

Image by Alberto Cairo on thefunctionalart.com

Inspired by this dinosaur, two researchers at Autodesk Research, Justin Matejka and George Fitzmaurice, created 12 datasets that have similar summary statistics as the dinosaur, but with different distributions and visual appearance when plotted on scatterplots. The scatterplots reveal very different shapes such as a star and a cross.

The authors developed a new statistical method for generating such datasets. As they explain in their paper, “new datasets are iteratively generated from a seed dataset through random perturbations of individual data points, and can be directed towards a desired outcome through a simulated annealing optimization strategy”.

The Datasaurus Dozen serves as a fresh approach to underscore the importance of data visualization. While summary statistics aid in understanding data, they fall short in revealing the whole story. It’s a notion that is often taught in statistics lectures with Anscombe’s Quartet: four scatterplots that, although being visually distinct, all have the same mean, variance and correlation coefficient.

Download the Datasaurus Dozen Dataset

  • Download the Datasaurus Dozen dataset from the Autodesk Research source here
  • Unpack the ‘DatasaurusDozen.tsv’ file from the SameStatsCode.zip archive.
  • Once retrieved, you’re all set to delve into its exploration using Python!

Exploring the Datasaurus Dozen’s Summary Statistics

Summary statistics are statistics that summarize the characteristics of the data and the relationship between its variables.

They are very useful since they reduce large number of observations down to a single numbers that are easily interpretable. Mean, standard deviation and correlation coefficient are example of summary statistics.

● Let’s analyze the summary statistics of the the Datasaurus Dozen using Python. To begin, we’ll import the dataset into a DataFrame using Pandas.

import pandas as pd

data = pd.read_csv('DatasaurusDozen.tsv', sep='\t')
data.head()

As observed, the dataset comprises a column labeled ‘dataset’ which contains the names of the datasets. Additionally, it includes two columns, ‘x’ and ‘y,’ representing the coordinates of individual data points.

● Let’s check how many datasets there are using the len() and unique() functions

len(data['dataset'].unique())

__
Ouput
13

There are indeed 13 datasets. The dinosaur and 12 another datasets.

Get Marc Bolle’s stories in your inbox

Join Medium for free to get updates from this writer.

● Let’s now retrieve their name:

data['dataset'].unique()

__
Ouput
array(['dino', 'away', 'h_lines', 'v_lines', 'x_shape', 'star',
'high_lines', 'dots', 'circle', 'bullseye', 'slant_up',
'slant_down', 'wide_lines'], dtype=object)

● Now, let’s calculate the summary statistics (mean, variance and standard deviation) of each dataset using Pandas’ functions groupby() and agg() :

datasets_grouped = data.groupby('dataset')
datasets_grouped.agg(['count', 'mean', 'var', 'std'])

As we can see, all datasets display similar summary statistics, each matching to two decimal places.

● Let’s now compute the correlation coefficients between X and Y in each dataset using the corr() function:

corr = data.groupby('dataset')[['x','y']].corr().iloc[0::2,-1]
corr

Once more, the datasets demonstrate analogous correlation coefficients, matching to two decimal places. Given the proximity of this coefficient to zero, we can infer a lack of substantial relationship between the X and Y variables.

● Similarly, we can calculate the covariance matrix for each dataset using the cov() function:

cov_matrix = data.groupby('dataset').cov()
cov_matrix

Once more, the thirteen datasets display the exact same covariance matrix, matching to two decimal places.

● Let’s now try to compute a linear regression equations for each of these datasets using the scipy.stats module:

Once more, they all have the exact same linear regression equation;

At this point, anyone looking at all of these similar statistical measures should conclude that all thirteen datasets have exactly the same distribution and appearance when displayed on scatter plots. Isn’t it?

Visualizing the Datasaurus Dozen in Python

●Let’s now visualize the 13 datasets in scatter plots using Python and Seaborn:

import seaborn as sns 

grid_scatterplots = sns.FacetGrid(data, col="dataset", hue="dataset", col_wrap=4)
grid_scatterplots.map_dataframe(sns.scatterplot, x="x", y="y")
Press enter or click to view image in full size

Here we are, 13 datasets with similar statistics but with dramatically different appearance. What a fascinating and curious phenomenon!

● Let’s fit a linear regression line on each dataset using seaborn’s lmfit() function:

sns.lmplot(data=data, x="x", y="y", col="dataset", hue="dataset", line_kws={'color': '#003f5c'}, col_wrap=4, ci=None, height=3)
Press enter or click to view image in full size

We can observe that all the datasets exhibit the same linear regression line as well.

● Even more interesting, we can see that, as the data points change shapes, they retain the same summary statistical values to two decimal places throughout the process:

Press enter or click to view image in full size
GIF by Autodesk Research

Conclusion

The playful revelation of a dinosaur shape in the Datasaurus Dozen serves as a whimsical reminder of the limitations inherent in relying solely on summary statistics. It underscores the crucial role of data visualization for a more profound understanding of data.

--

--

Written by Marc Bolle

📊 Data consultant exploring Tech 🚀🌟