A Seed Is Not Reproducibility

set.seed(42) fixes the starting state of one random number generator. Seven experiments in R on everything it leaves open.

Diogo Ribeiro is a mathematician, researcher and lecturer at the Faculdade de Media Artes e Design, Universidade Técnica do Porto.

Almost every R script that draws a random number begins with the same line, and the line is usually set.seed(42) or set.seed(123). It is taught as the thing one does to make an analysis reproducible, and it is often the only thing done. The belief behind it is that a program with a fixed seed is a deterministic program, so that anyone who runs it again will get the same output.

The belief is half right, and the half that is wrong is worth spelling out. The output of a stochastic program is a function of several arguments. One is the seed. The others are the rules by which the generator turns its state into numbers, the exact sequence of calls that consume those numbers, the order of the data the numbers are applied to, the way the work is divided among processes, and the floating-point arithmetic of the machine and its libraries. Fixing the seed fixes one argument of that function. The experiments below hold the seed fixed and move the others one at a time, and in every case the output moves with them.

There is also a second and different sense in which a seed fails to deliver reproducibility, and I think it is the more important one. Suppose all the…