Looking at Artemis — PINNs

Originally published at https://livnlearns.com — I would recommend reading there, cause latex didn’t import and I couldn’t be bothered :)

In Looking at Artemis we built Artemis I’s trajectory from scratch: Earth, J2, the Moon, the Sun, solar radiation pressure, lunar J2, and one integrator. After four days it missed JPL Horizons by 7,426 km. Not because the physics was wrong, but because Orion isn’t a passive rock. It fires its engines, and a forward propagator has no way of hearing about it.

In Differential Equations V we went through the theory of physics-informed neural networks (PINNs): a network that fits data, but gets penalized every time it breaks a differential equation.

This post puts the two together. The question for the proof of concept is small on purpose:

If we hide four hours of Orion’s trajectory, can a neural network that knows gravity fill the gap better than one that doesn’t?

This post covers what worked, what didn’t, and what I’d change. You can find the code here

Picking a clean arc

My first attempt used the full four-day Horizons window, and it went badly: the PINN did worse than a plain neural network with no physics at all, and its learned correction term grew to about 100× the Moon’s pull. A likely culprit was the trajectory correction burns. The physics loss assumes Orion is coasting, and the four-day window contains several thruster firings, not all of which were in my list of burn windows. Wherever the network was asked to obey coasting physics during a burn, it was being trained against the wrong equation.

So for the proof of concept we pick a 20 hour arc that sits between the first two Outbound Trajectory Correction burns: it starts about 45 minutes after OTC-1 and stops about 15 minutes before OTC-2. Every sample in it is a coast.

The data is Horizons’ state vectors for Orion (target -1023) relative to Earth’s centre, in the Earth-equator J2000 frame, every 10 minutes. That gives 121 samples. Over the arc Orion climbs from about 91,000 km to about 214,000 km from Earth.

Simulating a blackout

To test gap filling we hide the middle 20% of the arc (hours 8.0 to 11.8, 24 samples) and then train on every second sample outside it. That leaves 49 training points, about 40% of the arc.

Figure 1. The 20 hour coast arc in the XY plane, with the hidden gap in red (left), and Orion’s distance from Earth over the arc (right)

Physics, in Torch

The gravity functions are the same ones from the first Artemis post, rewritten as batched Torch functions so gradients can flow through them. For this arc the physics prior is Earth’s point mass plus J2.

Propagating Earth + J2 for one hour from the Horizons initial state lands 0.07 km from Horizons. The physics code from the first Artemis post carries over fine.

From a network to an acceleration

The network takes normalized time τ = t / t_scale, which runs from 0 to 1 over the 20 hours, and returns a position. Velocity and acceleration come from differentiating the network with autograd, with the chain rule converting back from τ to seconds:

Because the loss needs a second derivative of the network, the activation has to be smooth. We use SiLU rather than ReLU, whose second derivative is zero almost everywhere.

Baking in the initial state

So a standard Neural Network for our data would look something like:

i.e. a smooth function in tau

the problem is if we initialize it randomly and evaluate g_theta(0)​ we get whatever the random weights the network arbitrarily produces, not necessarily zero, not r0, not anything meaningful

A randomly initialized network, at tau = 0 returns whatever its weights happen to produce. Rather than adding a loss term to make it start in the right place, we build the starting position and velocity into the architecture

Simple MLP

We usea simple MLP as a baseline, i.e. a simple non boundary corrected mlp scaled to original units

Approximates trajectory by directly mapping normalized time and space

Centered MLP

To give an MLP baseline since its already at a disadvantage not knowing ICs — we create a centered mlp which predicts averages instead of raw number

Initial Conditions (Position)

Now we use PINNs, by implementing a simple boundary constraint

spatial trajectory (r) as a function of normalized time (tau)

Where N_theta(tau) is the raw output of the mlp

r_scale is a scalar used to non-dimensionalize the spatial domain and prevents vanishing/exploding gradients

At tau = 0, the second term tau . N_theta(tau) strictly evaluates to 0, forcing r(0) = r_0 regardless of the network’s weights

Evaluating at tau = 0:

The initial velocity relies entirely on the untrained, raw output of the network N_theta(0). The optimizer must waste epochs learning this value, creating severe stiffness at t=0

Initial State (position and velocity)

Now we proceed to next step — to have the true velocity at tau=0 as well

To force r(0)=r0, we needed the free term to vanish to zeroth order at tau=0 hence we multiply by tau¹ now to additionally force r’(0)=v0 we multiply by tau²

you can think of this as just 2nd term of the Taylor series

Constrain both the initial position r_0 and initial velocity v_0:

Residual

Now we apply the same idea but to accelaration,

The equation of motion is dominated by a term we couldn’t compute exactly — Newtonian gravity, plus known perturbation models (harmonics, third body). Real trajectories still deviate slightly from what that physics predicts, and Delta_a(tau) is the network’s job — absorb whatever acceleration is left over once the known physics has been subtracted

here we are trying to model Delta_a = f(tau)

i.e. residual accelaration as a function of normalized time — this is a non parametric trajectory spline fitter for a single, fixed arc (doesnt take into account anything else)

The loss

The loss is a data term on the 49 training points plus a physics term on 512 collocation times, redrawn every epoch. Collocation times that land inside a burn window are rejected, so the network is never asked to obey coasting physics while an engine is firing. Each term is normalised: positions by r_scale (the largest coordinate on the arc, 205,232 km) and accelerations by a_char = μ / r_scale² (9.46 × 10⁻⁶ km/s²).

In the MLP we compute a Normalized Data Loss: Computes the Mean Squared Error (MSE) scaled by r_scale

In our PINN, we have:

Where the last term is only in residual Network where we want to model the missing physics

Cosine annealing shrinks the learning rate over training so the optimiser can settle into the narrow valleys that competing data and physics terms create, and gradient clipping stops an occasional wild second derivative from knocking training off course.

To know whether the physics is doing anything, the PINN needs rivals. Each rung of the ladder tries one approach, and every model that learns from data sees the same 49 training points:

1. Curve fits (1a, 1b). A cubic spline and a degree-7 polynomial through the training points. No physics, no network, just fitting the curve — i actually wrote a bit more about spline here.

2. Data-only networks (2a, 2b). The same MLP size as the PINN, trained on positions alone. 2b centres and scales the outputs to give it a fairer start. This is data only networks — no physics.

3. Head start only (3). The PINN’s exact architecture with r₀ and v₀ built in, but with the physics weight set to zero. This helps us see how effective is the PINN architecture is on its own (ignoring the physics loss we use in PINNs)

Model 3 knows where the craft starts and how fast it’s going, and after that it’s just a curve-fitter. In the gap it has no data, so it draws a plausible smooth curve and misses by about 88 km.

4. PINNs (4a, 4b, 4c). 4a builds in only r₀ and has to work out v₀ itself. 4b builds in both. 4c adds a second small network δa(τ) that learns a correction acceleration for whatever Earth + J2 leaves out.

In 4a the network has to work out Orion’s starting velocity itself. Getting within 2.2 m/s of 2,595 m/s (0.08%) sounds excellent. But a velocity error turns into position error linearly with time, so errors that small add up fast:

• Rule of thumb: 1 m/s ≈ 3.6 km per hour.
• 2.2 m/s × 10 h (the middle of the gap) ≈ 79 km. The gap error is 88 km.

A trajectory with a wrong v₀ misses those late points by more and more, so the data should force v₀ to the right value. It doesn’t, because the data term is weighted so weakly

Model 4b has the same starting point but is also told “whatever curve you draw must accelerate the way gravity says — i.e. the PINNs loss” That rule reaches into the gap even though no data does, and it pins the curve to within about 5 km.

5. Classical propagation (5). No learning at all: solve_ivp integrates Earth + J2 forward from the true r₀ and v₀.

Overall

Figure 3. Position error over the whole arc for every model (log scale). The hidden gap is shaded red.

Get Winnie W.’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The physics term does real work. Model 3 and model 4b have identical architectures, identical data and the same built in r₀ and v₀. The only difference is whether the physics loss is switched on. Gap RMSE goes from 87.7 km to 5.5 km, 16 times better. That is the headline result of the proof of concept: the mechanics of a PINN (autograd accelerations, hard constraints, burn masked collocation) all work on real mission data, and the physics term measurably helps.

The data loss only ever looks at the 49 training times. The physics loss is checked at 512 random times per epoch, spread across the whole 20 hours, gap included. So the physics loss is the only part of training that supervises the gap at all. “Bending” is acceleration (the second derivative), which is why the network is differentiated twice.

The PINN beats the propagator in the gap. Classical propagation (solve_ivp) uses the same physics and the same start state and gets 9.0 km in the gap. The PINN gets 5.5 km, so the training points are adding information that pure propagation doesn’t have. Model 5 and model 4b have the same physics and the same start. The solve ivp knows the start, applies the physics, step forward, and never looks at the data again. The physics is missing the Moon and Sun, so a small error builds up steadily. The PINN starts the same way, but it has also seen dots out to hour 20. Those late dots act like a faint tug on the far end of the curve. That tug straightens some of the drift in the middle.

It can recover what it isn’t told. Model 4a is never given the starting velocity, and it works it out from the data and the physics to within 2.2 m/s of the true 2,595 m/s, an error of 0.08%. Because the position error from a velocity error grows in a straight line with time: 2.2 m/s over 10 hours is about 80 km of drift, which is close to 4a’s 88 km gap error. Orbital mechanics is unforgiving about velocity. A a wrong v0 makes the curve miss the hour 12–20 dots by more and more, so the data should correct it. It barely does, because the data term is too weak

The curve fits win. A cubic spline (1.55 km) and a degree-7 polynomial (1.11 km) both beat the PINN in the gap, and neither knows gravity exists.

Over 4 hours, the path is extremely smooth, very close to a parabola. There are points every 20 minutes right up to both edges of the gap. A cubic spline is built out of cubic pieces, so it can represent a near-parabola almost perfectly, its error comes from the tiny part of the curve’s shape inside the gap that a cubic can’t reproduce. The degree 7 polynomial does even better because it’s fitted to the whole arc, so it “knows” the global shape of the curve. The spline isn’t clever. The test is simply easy for it — dense, clean, smooth data and a short gap.

Adding a learned correction made things worse. Model 4c, the PINN with δa, goes from 5.5 km to 41.2 km. And δa is not learning the physics that’s missing. The notebook checks it against the Moon + Sun acceleration that Earth + J2 leaves out, The two diagnostic numbers:

• Size, 2.8× too big. δa isn’t carrying the Moon + Sun pull (1.9 × 10⁻⁸). It’s soaking up the network’s own wobble, which is about 10⁻⁷, so it lands in between (5.4 × 10⁻⁸).
• Direction cosine +0.21. This measures whether two arrows point the same way: +1 is the same direction, 0 is at right angles (unrelated), −1 is opposite. +0.21 means about 78° apart, so δa is mostly unrelated to the real Moon + Sun pull.

Once δa absorbs the disagreement, the physics stops constraining the curve, and 4c slides back towards the no-physics mode

4c sits between the other two, and its train error is basically model 3’s. That’s the evidence the physics got partly switched off.

The data-only networks miss their own training points. Models 2a and 2b are 118 to 359 km off on the very points they were trained on.

The final loss of 10⁻⁶ looks tiny. But positions are divided by 205,232 km, so 10⁻⁶ means 0.1% per coordinate, and 0.1% of 205,000 km is about 200 km, the hard-IC architecture gets the starting point and starting velocity exactly right for free, so the network has less to learn.

Figure 4. Zoom on the gap with a straight-line trend removed from each axis, so the curvature of the arc is visible. The curve fits and the PINN (4b) sit on the truth at this scale; the data-only networks and 4a are tens to hundreds of km off.

1. The PINN is a propagator in disguise

Once again there’re 2 ways to fill a gap:

• Interpolating: use data on both sides of the gap and bridge between them. A spline does this. Its errors are near zero at data points and largest in the gap.
• Propagating: start at t = 0 and step forward using physics. solve_ivp does this. Its errors start at zero and grow steadily with time, whether or not there's data.

A PINN is supposed to interpolate using physics as the bridge. This one behaves like a propagator.

Look at Figure 3 again. From about hour 15 onwards, the PINN’s error curve lies on top of the classical propagation curve, and both end the arc near 40 km (38.0 km for the PINN, 40.9 km for solve_ivp). A blindfolded walker’s error grows smoothly with time. solve_ivp is the pure blindfolded walker. From about hour 15, the PINN's error line merges with it, and both end near 40 km. If the PINN were really using the data at hours 12-20, its error there would stay small, because those are points it was trained on.

Figure 5. Train RMSE (hollow) against gap RMSE (filled). An interpolator should do worse on points it has never seen; every physics-driven model does the opposite. (The original MLP is left out: its train error is dominated by a 975 km miss at the very first sample.)

For an interpolator, error on the hidden points should be worse than on the training points. Every physics-driven model shows the reverse. That is the signature of a model integrating forward from t = 0: its error grows with time, and the gap happens to sit earlier in the arc than half the training points. The PINN starts from the exact r0 and v0, integrates Earth + J2 forward, and the training points at the far end barely pull it back.

The reason is in the loss scales. Each term is normalised by the size of its signal, not the size of its expected error. A 1 km position error costs about 8 × 10⁻¹² in data loss; the PINN’s final data loss of 2.6 × 10⁻⁹ corresponds exactly to its 18.2 km training RMSE. Now, the two ways the network could disagree with something:

Figure 6. The two ways the network can disagree with something, priced in the loss units.

The loss adds two different kinds of error: position error (km) and acceleration error (km/s²). Wecan’t add km to km/s² directly, so the normalization quietly sets an exchange rate between them. Dividing positions by 205,232 km and accelerations by 9.46 × 10⁻⁶ km/s² means:

1 km of position error costs the same as 4.6 × 10⁻¹¹ km/s² of acceleration error (9.46 × 10⁻⁶ ÷ 205,232 = 4.6 × 10⁻¹¹.)

Now price the Moon and Sun at that rate. Their pull is 1.9 × 10⁻⁸ km/s², and 1.9 × 10⁻⁸ ÷ 4.6 × 10⁻¹¹ ≈ 420 km.

So as far as this loss is concerned, “admit the Moon exists” is as expensive as missing the data by about 420 km. The alternative, ignoring the Moon and following Earth + J2, only misses the data by about 19 km. The optimizer takes the deal. It isn’t malfunctioning, it is minimizing the loss we gave it.

A cleaner way to set the weights: dividing a squared residual by σ² is a Gaussian log-likelihood with standard deviation σ. So the current loss implicitly claims positions are trustworthy to ±205,000 km and the gravity model to ±9.5 × 10⁻⁶ km/s². More honest numbers would be about 1 km for positions and about 2 × 10⁻⁸ km/s² for Earth + J2 (the size of what it leaves out) i.e. λ_p ≈ 6 × 10⁻⁶ rather than 1, a shift of more than five orders of magnitude towards the data.

2. The prior keeps the small term and drops the big one

Figure 7. Acceleration budget on the arc. Earth terms are evaluated at mid-arc (about 160,000 km); Moon + Sun and the network quantities are medians or RMS values from the notebook; solar radiation pressure uses the notebook’s Orion area, reflectivity and mass.

At 91,000 to 214,000 km from Earth, J2 is tiny, around 10⁻¹⁰ to 10⁻¹¹ km/s². The Moon and Sun, which the prior leaves out, are 50 to 1,500 times larger along the arc. A constant 1.9 × 10⁻⁸ km/s² acting for 20 hours displaces Orion by ½at² ≈ 50 km. That is the same size as classical propagation’s 40.9 km miss, and because of that, the PINN inherits that bias. More is explained in the Artemis I post.

It is tempting to call the Moon negligible for the first day because its pull is tiny next to Earth’s. Next to a km-level accuracy target, it isn’t. N-Body acceleration would make this better

One more number in that chart, the PINN’s own physics residual (RMS about 1.0 × 10⁻⁷ km/s²) is larger than the Moon + Sun signal it is missing. So the physics loss isn’t yet converged tightly enough to tell the missing physics apart from its own fitting noise, which brings us to the optimizer.

3. A free correction term makes the physics optional

If δa can be any function of time, the network can always set it to exactly the leftover. Then the check passes for any curve, right or wrong. That’s what “δa = r̈ − a_model(r) satisfies the physics loss perfectly” means: r̈ is the curve’s own acceleration, and δa just fills in whatever is missing.

Ways to make δa identifiable: put the known Moon and Sun into the prior first so δa only has to find leftovers at the solar radiation pressure scale (a few × 10⁻¹² km/s²), as mentioned above as well the Nbody accelaration, we used in first notebook — I’ll follow up on that one in a separate final post.

4. Adam alone stalls

In PINNs, the two loss terms pull in different directions, and those terms differ in size by orders of magnitude (data around 10⁻⁹, physics around 10⁻⁵).

The data-only networks’ losses flatten out by epoch 1,000 to 1,500, while the cosine schedule still has the learning rate at half to three-quarters of its starting value, so they have stalled rather than run out of step size.

The centred MLP’s loss changes by 1% over its last 500 epochs, and the PINN’s physics loss improves by less than 3× over its last 2,000.

The data-only networks miss their own training points by 118 to 359 km, while a degree-7 polynomial fitted by least squares misses them by 0.36 km.

The original MLP’s error has equal lobes of about 400 km that cross zero near hours 1.7, 8.5 and 17.5: the network has captured the smooth, low-order shape of the arc and stalled there.

In the PINNs post I described a two-phase optimisation: Adam to find the right valley, then L-BFGS to settle to the bottom of it. This notebook only runs the first phase. Adding L-BFGS is probably the cheapest improvement on the list.

The model isn’t the only thing with problems. A few things about the test setup limit what these results can say.

This is the spline’s home turf. The data is noise-free and smooth, there’s a training point every 20 minutes, and the gap is only 4 hours. Horizons isn’t raw tracking data either; it’s already a smooth trajectory product. Every one of those conditions favors a spline. The places where a spline struggles are exactly where physics should help:

• Noise. A spline passes through every point, so it copies the noise and wiggles. Gravity doesn’t wiggle, so a physics loss smooths the noise out.
• Sparse data and long gaps. A spline’s error across a hole grows roughly with the fourth power of the hole’s length. A well-weighted physics bridge grows with the square. Short gaps favour the spline; long gaps should favour physics.
• Forecasting. Past the last data point a spline has nothing to go on. Physics keeps going.

In other words, this proof of concept tested the PINN where it’s least needed.

The PINN is handed the exact starting state. Models 3, 4b, 4c and 5 are given Horizons’ r0 and v0 exactly. In a real blackout you’d only know the starting state as well as the tracking before the blackout allowed. Model 4a shows what that costs: 2.2 m/s of velocity error is about 80 km by the middle of the gap.

Where the gap sits matters. A propagator’s error grows with time since the start. A gap at 40–60% of the arc catches the PINN while its error is still 4–9 km. The same gap at 60–80% would land where its error is 9–25 km. A spline doesn’t care where the gap is. Sliding the gap across the arc would make the comparison fair.

One run, one seed. Every number here comes from a single training run. Neural networks start from random weights, and a different start can give a different answer, so there are no error bars. On top of that, the 4b cell doesn’t reset the random seed the way the other runs do, so models 3 and 4b didn’t start from identical weights. A 16× difference is unlikely to be luck alone, but several seeds per model would settle it.

The classical ceiling isn’t one. Classical propagation was given the perfect starting state and never looked at the data. That isn’t the best a classical method can do with the same physics and data, and the PINN already beats it in the gap. The fair rival is batch least-squares orbit determination. It asks: which six numbers (r₀ and v₀), propagated with my physics, best match all 49 points? Because it uses points at both ends, the error from the missing physics gets spread across the arc instead of piling up at the end. That’s what model 4a was trying to do with a neural network; orbit determination does it directly. It’s also the closed-loop approach flight dynamics teams actually use, as we saw at the end of the first Artemis post.

1. Rebalance the loss. Sweep λ_p across 1, 10⁻², 10⁻⁴ and 10⁻⁶, and try the value from the exchange-rate argument (about 6 × 10⁻⁶). Too much physics weight gives a propagator that inherits the model’s bias. Too little gives a data-only network that can’t bridge the gap. The best value sits in between.
2. N-body propagator as mentioned previously. It should remove the 40–50 km bias from the missing third bodies.
3. Run Adam first, then L-BFGS on a fixed set of collocation points. The points need to stay fixed because L-BFGS works out the shape of the valley from its last few steps, and resampling the points every step would keep reshaping the valley under it.
4. Adding a batch least-squares orbit determination, with the same physics and the same 49 points, as the baseline to beat.
5. Add position noise (1 km and 10 km), thin the data to one point per hour, widen the gap from 20% to 60%, forecast the last 20% of the arc, and give v₀ a realistic error. Each of these targets a spline weakness, or the head start of knowing the exact state. This would even out the spline advantage.
6. Running at least five seeds per model, and sliding the gap across the arc, get error bars.
7. Use a constrained δa (for example a constant vector) on top of an n-body prior, with burn windows from both the public burn list and the Δv-residual detector. On this 20-hour arc, once the Moon and Sun are in the prior, only about 15 m of solar radiation pressure is left for δa to find. The burns are where δa has something real to learn.

I guess my goal was applying PINNs to this problem which i discussed before here — what looked like a simple curve — and here is what i learnt:

As a proof of concept, the pieces work. A neural network can be differentiated twice into an acceleration and checked against gravity. The starting state can be built into the architecture instead of requested through the loss. Collocation can skip over burn windows. And switching the physics term on makes the same network fill the gap 16 times better. On real mission data, that isn’t nothing.

But it isn’t yet what a PINN is meant to be: an interpolator that uses physics to bridge the gap. In its current form it’s a propagator, an expensive numerical integrator of an incomplete force model. Four things put it there:

• The loss weights make ignoring the data about 480 times cheaper than ignoring the Moon.
• The prior keeps J2, which barely matters this far out, and drops the Moon and Sun, which do.
• The correction term has enough freedom to make the physics optional.
• Adam stops before reaching the bottom of the valley.

On clean, dense data with a short gap, a cubic spline beats all of it.

The good news is that every one of those problems is specific and testable, and a couple come with numbers attached. Next time: rebalance the loss, add the Moon and Sun, finish the optimisation, and race orbit determination instead of a blind propagator. Then move to where a spline can’t follow: noisy data, sparse samples, long gaps, forecasting, and eventually the burns themselves.

Looking at Artemis — Simulating the Artemis I Trajectory from Scratch Differential Equations V — Physics Informed Neural Networks (PINNs) JPL Horizons API Code — MLSandbox — Winnie

Originally published at https://livnlearns.com on September 28, 2026.