The Hidden Cost of Application Parameters

Maybe you know the feeling. An interesting programming project is in front of you. Your boss or PhD supervisor has given you a task, or perhaps you simply want to start something new in your spare time.

You might want to write a scientific numerical solver, create your personal website, or develop an application that could make everyday life easier for millions of people. You already know which technologies you would like to use. The mathematical equations for your solver have been discretized, you’ve collected the necessary resources from your collaborators, and you have a reasonably clear picture of the software architecture. Perhaps you’ve even thought through the graphics, the GUI layout, and how everything should work together.

And then you start…

The first parameters

One of the first things you need to do is define your settings, constants, and constraints.

You begin with hard-coded parameters in your source files. Programming is spontaneous and fast. Perhaps you need to set the sizes of your GUI panels, choose the colours of your plotting components, or fine-tune your numerical solver using interpolation parameters and convergence criteria.

Things get more interesting when you have to integrate external resources: text files, CSV tables, binary data, or whatever your customers and collaborators have provided. The data may change several times during development because your colleague has only prepared a preliminary version for testing.

Meanwhile, parameters begin to travel from one source file to another. Sometimes they are runtime variables, sometimes compile-time constants. Often, you eventually consolidate them into separate files because several parts of your application need to access the same information.

Maybe you’ve already created a basic YAML configuration file.

The code structure is constantly changing in front of your eyes. New concepts appear, old ones disappear, and you keep adjusting the architecture as you encounter problems, bugs, and unexpected requirements.

At this point, you are probably not particularly worried about the organization of your parameters. You know what they mean, where they are defined, and how they interact. After all, you have just written the code yourself.

And those units and numerical types!

Every scientifically oriented programmer knows that not all numbers are interchangeable.

A density expressed in grams per cubic centimeter is not numerically the same as one expressed in kilograms per cubic meter. Both can describe exactly the same physical quantity, but only if the conversion is handled correctly.

Similarly, calculations performed using 16-bit unsigned integers behave very differently from those using 64-bit floating-point numbers. Integer ranges, numerical precision, rounding errors, and overflow can all affect the results.

A simulation can run perfectly well without reporting any errors, even though an incorrect unit conversion or an unsuitable numerical representation has changed the physical meaning of the problem.

Some numerical errors accumulate during repeated calculations. Others become amplified through unstable algorithms or poorly conditioned problems. Sometimes you discover the consequences only after hours of wasted computational time.

And then there are the dependencies between parameters.

A numerical time step may depend on the spatial resolution of your computational grid. An interpolation method may require a certain number of neighbouring points. Some settings only make sense when a particular physical model is enabled.

Individually, none of these requirements is particularly mysterious. The difficulty is keeping them consistent as the application evolves.

Consider a simple configuration:

simulation:
  time_step: 0.01
  duration: 10.0
  output_interval: 100

Nothing unusual here.

But what exactly is time_step? Is it expressed in seconds, milliseconds, or something else? Can it be negative? Does it have an upper limit? Is that limit related to the spatial resolution or another numerical constraint?

And does output_interval mean every 100 simulation steps or every 100 seconds?

Of course, you can answer these questions in your documentation or application code. You can also introduce a schema, a validation library, or a more sophisticated configuration system.

But now you have several places where the meaning of your parameters may need to be maintained.

The analysis scripts come next

Eventually, the main code is running, and it is time to analyze your results.

You create one Python script, Bash pipeline, or Jupyter Notebook after another. Each has its own settings. Sometimes you need a parameter from your main application, but its definition is buried somewhere in the source code.

Rather than extracting and reusing it, you simply define it a second time in your analysis script.

The scripts change, branch into different versions, and gradually develop their own configuration requirements. You modify something in the main code and realize that you also need to adjust a few things here and there.

Perhaps the analysis expects a particular output format, a specific physical unit, or an array with a certain shape. Perhaps it relies on a numerical constant that you have already defined elsewhere.

You can certainly keep track of these things. But every additional copy of the same information creates another opportunity for inconsistencies.

And none of this is necessarily bad programming. Often, it is simply the fastest way to make progress when you have a scientific problem to solve and limited time to solve it.

Finally, your project is finished!

This whole machinery continues to run, and eventually, you are done. At least for now.

You have completed your task, demonstrated your results, and satisfied the initial goals set by your supervisors or collaborators. Maybe you have even published an interesting paper about the project in a respected scientific journal.

People are interested in your work. Perhaps somebody wants to use your software.

Great!

And then it starts again…

Your project should now serve not only you but also other people.

You know how everything works because you have spent months or even years developing, debugging, and optimizing the code. You know which parameters can safely be changed, which combinations make sense, which parameter options are permitted, and which settings should never be touched without understanding their consequences.

Get Ondrej’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

But what about your colleagues?

What about somebody who discovers your project on GitHub and wants to try it?

Or the new person who has just joined your company or research group for a semester-long internship?

You know where to look in the source code, which parameter to adjust, and which numerical constant came from which scientific paper. You can explain everything.

And perhaps you will have to, because much of this knowledge still exists primarily in your head.

Unfortunately, you are already thinking about your next project.

The knowledge needs to leave your head

Now the project needs documentation.

You might use Sphinx, Doxygen, or simply write a comprehensive README using Markdown. You go through your source code and describe what each parameter means, which values are allowed, which other parameters it depends on, and where the relevant numerical constants come from.

Furthermore, you document the expected input files, the units, the default values, and the assumptions behind your calculations.

Then you begin thinking about validation.

What happens when somebody enters an invalid value? What if they provide data in the wrong units? What if they enable two mutually incompatible options?

You may want to make the software safer for somebody who has not spent the last two years working on it.

Some of these checks already exist in your code. Others have never been implemented because you always knew which values to use.

You begin adding error messages, validation rules, and more documentation.

Perhaps you even create a graphical configuration editor so that users don’t need to edit text files manually.

And once again, you find yourself describing the same parameters, their relationships, and their meaning.

Only now you are describing them for a different purpose.

Haven’t we solved this already?

In many ways, we have.

There are excellent tools for managing structured data and application configuration. YAML, JSON, and TOML provide convenient ways to store settings. JSON Schema and validation libraries such as Pydantic can define types, constraints, and relationships between values. Pint can handle physical units, while systems such as CUE and Pkl offer more sophisticated ways to express and validate configurations.

These tools solve real problems, and for many applications they are entirely sufficient.

But a question remains.

Suppose your scientific application uses a YAML file for configuration, a Python library for validation, another library for unit conversions, and a documentation generator to explain its parameters.

Your numerical solver might be written in C++ or Fortran, while your analysis scripts use Python and you are orchestrating your environment using Bash or Zsh. Perhaps you also want a web interface for configuring simulations or an automated tool that inspects the available parameters.

All these components need to agree on what each parameter means.

You can certainly build such a system using existing technologies. But depending on the tools you choose, you may still need to write and maintain the interfaces connecting them.

The same parameter knowledge can end up distributed across several representations: its name and value in a configuration file, its numerical type in the application, its physical unit in a conversion library, its constraints in validation code, and its explanation in the documentation.

The difficulty is not necessarily storing the parameters.

It is keeping their meaning consistent throughout the software.

What if the parameter itself carried this knowledge?

What if defining an application parameter meant more than assigning a value to a variable?

Imagine that you could declare its numerical type, physical unit, dimensions, default value, constraints, dependencies, and documentation in one place.

That definition could then serve as a common interface for different parts of your project.

Your compiled numerical code could consume the parameter values. Your Python scripts could access the same definitions. A configuration editor could discover which inputs are available, and documentation could be generated directly from the declarations.

Even an AI agent could inspect the parameter definitions and learn which values are allowed, which units are expected, and which dependencies need to be respected, without having to infer everything from scattered source code.

Of course, this does not mean that existing tools cannot accomplish these tasks. Many of them already provide important parts of such functionality.

The interesting question is whether we could make this process more systematic, reusable, and less dependent on application-specific integration code.

In other words, could an application’s parameters become an explicitly defined interface rather than a collection of settings scattered throughout the software?

And could we build that interface once, then reuse the same approach for another application without having to reinvent the entire machinery?

This is the idea that eventually led me to experiment with a declarative, unit-aware parameter language called DIPL, as part of the open-source SciNumTools project.

For example, DIPL allows a parameter to be declared with its hierarchical name, numerical type, array shape, value, physical unit, constraints, dependencies, description, and scientific reference in a compact, declarative form:

simulation
  max_velocity float64 = 2000 cm/s
    ?doi "10.1016/j.oceaneng.2016.12.003"

boundary[inlet]
    velocity float64[3] = [5.0, 0.0, 0.0] m/s
      !condition ({.} <= {?simulation.max_velocity})
      ?descr "Inlet velocity vector"

The goal is not to replace every existing configuration format or validation library. It is to investigate whether putting more of the parameter knowledge into a single, inspectable definition can simplify the development and maintenance of scientific applications.

Whether this approach is genuinely more convenient than the established alternatives is a question that deserves practical examples, comparisons, and benchmarks.

But after repeatedly encountering the same problems in scientific software development, I think it is a question worth investigating.