Do Not Touch These Loops: A Journey Through Scientific Source Code
The warnings, jokes, and forgotten conversations that reveal the people behind the simulations
A scientist wishes to model the universe. This is an ambitious project, but let us assume the equations are ready.
The next step is to explain them to a computer. The computer, to its credit, is willing to simulate stars, molecules, and the atmosphere. It does, however, have some preferences about which compiler you use, how you initialize your variables, and whether you have remembered a semicolon.
I searched the source code and Git history of projects listed in SNT Hub for traces of those negotiations: comments that warn, wonder, joke, and occasionally address the future in capital letters. Together, they tell a story about the people behind the calculations, and what happens after an elegant idea meets an actual machine.
In the beginning, there were settings
Before a simulation can get to the important business of simulating, someone has to choose its initial values. This sounds straightforward until the values begin interacting with one another.
In Castro, an astrophysical simulation code, an initialization routine checks two density settings. A comment explains why: “if not, funny things can happen above the atmosphere”.
The code raises an error if `small_dens` is at least 99% of `low_density_cutoff`. The program appears to prefer keeping the funny things a mystery.
Other values keep a calculation away from a mathematical difficulty. In MOOSE, a velocity calculation uses `1e-42` when velocity is close enough to zero, avoiding a problem with automatic differentiation. The comment explicitly credits *The Hitchhiker’s Guide to the Galaxy*. Douglas Adams’s answer to life, the universe, and everything has acquired a minus sign, an exponent, and a job in fluid dynamics.
Arepo also assigns 42 to dummy group counts (link access may require an account) that a function expects, but the calculation does not use. Its comment doesn’t identify a literary reference, but the exchange is familiar: “I don’t need this number.” “The form requires a number.” “Fine. Forty-two.”
Time needs a value too. In WRF, the weather modelling system, an interval called `forever` is set to `1700000000` seconds — about 54 years. The comment calls it a magic number representing forever.
Infinity can be difficult to store; a large number can stand in when the surrounding code agrees on the convention. Still, it is comforting to know that WRF’s forever has a finite attention span.
The compiler would like a word
With the settings in place, the simulation must be built. Compilers have their own preferences.
A Castro commit from 2017 describes a “magic incantation” for configuring PGI alongside newer GCC headers on a particular machine. Anyone who has watched a build suddenly succeed after changing a path knows the temptation to write down the spell.
A 2015 WRF commit describes a pointer that should have been cleared after its memory was freed. The author imagined a future mystery blamed on a “random cosmic ray”. The patch offers an earthly explanation: the free function was clearing its local copy, so it was changed to clear the caller’s pointer instead.
Quantum ESPRESSO provides the sequel. In 2024, a change described a compiler issue with a matrix reduction and introduced a scalar reduction as a workaround. Later that day, a second commit changed an assignment to an accumulation. Its subject:
“Workaround was not working”
There is an entire engineering meeting in those four words.
The numbers are running. Please do not improve them yet.
Once the code builds and the calculation runs, it can visit places that sound more glamorous than they look in a debugger. SWIFT, an astrophysical simulation code, sends a density field through a forward Fourier transform with the comment “go to magic-land”. In that transformed space the code applies a Green function; later, an inverse transform brings it back. A 2018 commit added a version of the file with both comments.
Magic-land turns out to have a fairly strict immigration policy: you must arrive by Fourier transform and leave by the inverse. Between the two, useful mathematics happens.
Geant4, a toolkit for simulating particles passing through matter, approaches the supernatural more decisively. In the documentation for a triangular-facet intersection routine, a comment calls `goto` statements the “Work of the Devil” and says they were exorcised. Nearby, the documentation also gives a more conventional warning: the calculation expects a unit direction vector.
Sooner or later, another character enters our story: the helpful person who sees an opportunity to optimize the code.
In LAMMPS, a molecular simulation package, a set of loops comes with a welcome sign:
// DO NOT TOUCH THESE LOOPS!
The next two lines say attempted optimizations had performed worse in benchmarks. The surrounding calculation involves electrode atoms and grid weights. A 2022 refactor added the warning with the restructured loops.
It starts as a haunted-house warning and ends as a miniature experimental record.
Elsewhere, NWChem’s history appears to offer the opposite advice. A 2023 commit is titled “remove the loops — DGEMM will always be better”. Its patch replaces explicit nested loops in coupled-cluster kernels with calls to DGEMM, a matrix-multiplication routine.
These are different calculations, and the NWChem title is a contributor’s claim rather than a universal performance law. Together, they make an excellent welcome packet for anyone seeking optimization advice: preserve the loops, remove the loops, and please benchmark the ones in front of you.
CP2K has its own emphatic “Do not touch !!!!” above notes about numerical integration points. Same tone, different code; the reason for this warning remains a question for another day.
Get Ondrej’s stories in your inbox
Join Medium for free to get updates from this writer.
Not every message to the future comes with an answer. Another LAMMPS comment lists settings for cases with and without deformation or a thermostat and admits the writer is “not sure why these settings work best”.
The question is preserved where someone investigating the choice can find it.
Occasionally, the code comes with its own frank review. OpenFOAM’s documentation describes a utility that forces creation of things patches might use, then observes “This is fairly horrible.” MOOSE, before checks in a coupling-matrix integrity test, is more emphatic: “Yes, this is horrible. Don’t ask why…” Software documentation sometimes includes an emotional appendix.
Hidden rooms and final bosses
Beyond the warnings are a few places with names that sound at home in an adventure game.
MITgcm has a place explicitly called the “Secret Hiding Place”. It holds earlier values for atmospheric-model diagnostics, so the code can compare them with what comes later. The name arrived with the common block in 2006.
NWChem has an “Ugly secret option for now”: a diagnostic that can print slices of a complex absorbing potential. It was already called that when the feature was added in 2018. “For now” is one of the more flexible units of time in software.
GROMACS has a more theatrical secret identity. Its `bromacs()` function can choose an alternative expansion of its name, including “GROwing Monsters And Cloning Shrimps.” One can appear as the title of an output file if no title was supplied. In an early 1997 revision, the playful mode required an environment variable named `IAMCOOL`.
The historical record also reminds us that even jokes need maintenance. A 2020 GROMACS patch reports that choosing a quote could freeze on some systems when hardware random-number generation misbehaved. A follow-up addressed generator initialization. The program could get stuck deciding how to be funny.
The next room contains a choice between two ways of writing Fortran.
In NWChem, a comment announces “WARNING: This code is EXTREMELY EVIL!” The author then explains the creature: what looks like one function is used in different enclosing routines for different data types. The alternative, the comment says, would have been a forest of conditional compilation to change names throughout the code.
The warning appears in nine related files. A 2014 patch brought it to three of them alongside the change to contained functions.
NWChem also has an “Evil twin of task_optimize”: a copied and modified saddle-point optimizer dating back to a 1997 commit.
The monsters also announce themselves through error messages. GROMACS warns that unusual bonded-atom distances may mean “your system is exploding” — the simulated system, thankfully. A Geant4 diagnostic for an event with unfinished sub-events ends with “PANIC!!!” And when vectors fail an orthogonality check, GROMACS can report “DEATH HORROR: vectors not orthogonal”.
MITgcm provides a quieter ending to the level. After initializing sea-ice fields, a comment says “Now lets look at all these beasts” before conditionally plotting fields and masks.
Meanwhile, in the comments
No expedition is complete without a conversation between the people holding the map. In scientific software, that conversation sometimes takes place in the margins, with years between replies and a calculation waiting patiently below.
MITgcm preserves a conversation across time: a note beneath an explanation about boundary velocity points says “no idea what i was talking about ^^^^”. The next reader inherits both the explanation and its refreshingly honest review.
GROMACS has a similar exchange in a trajectory routine. One comment explains a time adjustment; the next begins “I don’t understand the comment above”, then says the cases the writer tried seemed to work. A 2008 patch moved the existing explanation and added the response beneath it.
In WRF, the discussion left a more visible mark on the calculation. An aircraft-observation routine contains a comment disagreeing with throwing away the original flight-level information. The 2014 patch that added the note also commented out the two assignments that would have marked those values as missing.
A postcard from the atmosphere
After all this work, our imaginary simulation deserves a holiday. Conveniently, WRF has already picked a destination.
In a domain-marking routine, a field is assigned `9021000`. The comment identifies the number as “Beverly Hills * 100”. A 2004 commit added the assignment alongside a comment saying the field is modified so the location of a nested domain (a smaller, finer-resolution grid within a larger one) can be seen.
A famous postal code became a marker the software can use and a human might remember.
In the interest of equal embarrassment
At this point, you might reasonably suspect that the author of this article has been inspecting other people’s repositories from the safety of a spotless codebase. I regret to report that my own project, SNT3, was also available for inspection.
SNT3 is my supporting tool for structured scientific inputs and quantities, rather than one of the simulation codes surveyed above. It is close enough to my keyboard to make a fair subject for the same treatment.
Its parser contains a TODO explaining that regular expressions are “not efficient, but fast to implement.” This is a performance optimization in which the thing being optimized is the programmer’s afternoon.
The uncertainty code has also given me a note to myself. In an earlier version, both `min` and `max` carried a TODO describing their uncertainty handling as “fishy”. Apparently, even the uncertainty code came with a note expressing uncertainty.
The note has since led to clearer documentation and a test. SNT3 now also offers Monte Carlo propagation, and uncertainty propagation is covered by SNT Bench. Future me did eventually read the note — and even did something about it.
What the code remembers
A paper usually tells us what the researchers concluded. The code can preserve a different part of the story: what they tried, what surprised them, which assumption needed a warning, and which question was still open when it was time to move on.
Long after a paper has presented the answer, someone may open a source file and find the conversation that helped make the answer possible. Every so often that conversation begins: “DO NOT TOUCH THESE LOOPS!”
If you maintain scientific software and have a favourite comment, workaround, or commit with a story behind it, I’d love to hear it — with the context that makes it worth preserving.
