Beyond PINNs: A Map of Modern Neural Network Paradigms

Eleven families of models that change what is being learned — not how many layers do the learning

Most introductions to deep learning present the same object. A neural network is a parameterised function

f_θ : ℝⁿ → ℝᵐ

fitted by minimising an empirical risk over observed pairs,

L_data = (1/N) Σᵢ ‖ f_θ(xᵢ) − yᵢ ‖².

Everything else — depth, width, convolution, recurrence, attention — is a question of how to parameterise that map efficiently and how to optimise it without the gradients dying on the way. This framing has been remarkably productive. It is also why physics-informed neural networks tend to feel slightly disorienting the first time you meet one.

A PINN keeps the network but changes the objective. Instead of asking only that the model agree with data, it asks that the model satisfy a differential equation. The loss becomes something like

L = L_data + λ_PDE · L_PDE + λ_BC · L_BC,

where the second term penalises violation of the governing equation and the third penalises violation of the boundary or initial conditions. Suddenly the network is not just interpolating a table of measurements. It is being pushed toward a function that is consistent with a mathematical description of the world, and that description was known before any data arrived.