The Road Less Travelled: Venturing beyond the Normal

Intuition Behind Fat Tails, Student’s t, Mixture Models, and GMMs

“Lehman Brothers just collapsed. There is no work to do and most of us are just wandering around aimlessly” read the terse message from a colleague who had left our Mumbai fintech firm just a couple of months prior to join the banking giant. He wanted to come back.

It was 2008, and the events unfolding on Wall Street were about to shake the global economy to its core. Nassim Nicholas Taleb’s newly popularized term, the Black Swan, was suddenly on everyone’s lips. This was undeniably a rare, catastrophic event.

But exactly how rare?

To understand that, we have to rewind back a few months to 13 Aug 2007 when a rather startling statement was made by Goldman Sachs’ CFO David Viniar to the Financial Times. He stated that the firm’s quantitative risk models had just witnessed “25-standard deviation events, several days in a row.” A six-sigma(σ) event is six standard-deviations away from the mean. It has a 2 in a billion chance of occurrence. A 25-sigma event is so statistically impossible that it shouldn’t happen even in a trillion trillion (repeated several times) years. To see it happen multiple days in a row is mathematically absurd.

What was going on?

The 2008 financial meltdown wasn’t just a failure of banking regulations; Among other things, it was also a catastrophic breakdown of a mathematical assumption: that the world of risk fits under a normal distribution which, by nature, is thin-tailed.

When Normal isn’t the Norm

Let us revisit the classic human height example of the previous article. If you plot the heights of a random pool of people of the same sex, a neat, symmetrical bell curve manifests itself.

Source: https://cs-courses.mines.edu/csci261/archive/Fall2015/lab07A.html

But does real-world height data follow the Gaussian Probability Density Function (PDF) to the letter? The answer is a surprising No.

Consider the absolute extremes. The shortest and tallest adult heights are roughly 21.5 and 107 inches. If you plug these numbers into our Gaussian PDF formula using the average human height, assuming a standard deviation of roughly three inches and calculate the tail area, it turns out to be a crazily low number… far greater one in a trillion trillion. Statistically, these people shouldn’t exist. Yet, on a planet of eight billion people, they do.

So it looks like the Gaussian could be under-reporting the probability around the extremes. This may be ok when calculating heights, but what happens when we use a Gaussian to model prices of financial instruments?

Sidenote: But why doesn’t the human height follow the exact Gaussian PDF formula? Why doesn’t it follow the Central Limit Theorem right down to the last decimal? Well, the data around the extremes involve complex genetic interactions and biological ceilings. A deeper dive is beyond the scope of this article.

Photo by Matt Howard on Unsplash

Why the Bell Curve Fails in Finance

The range of a Gaussian extends from negative infinity to positive infinity. Just as in the case of heights, using the Gaussian to model financial instruments leads us to the potential of negative pricing which is crazy. A bigger concern is the under-reporting of probabilities around the extremities!

The tails of a bell curve are the regions away from the peak on the left and right sides. Just like human heights plot, the tails here too decay fast and become way too thin (near 0), making the probability of extreme events much lower than reality! Unexpected events like a bank bankruptcy or wars are not rare. These shake the financial equilibrium. Pricing of financial assets needs to consider the possibility of these events happening at a greater frequency than what a Gaussian suggests! Fat tailed distributions are an alternative!

Modeling Markets with the Lognormal Distribution

Even before 2008, quants recognized the limitations of the standard normal distribution and often adopted a lognormal distribution for asset prices. By assuming that the natural log of the asset price follows a Gaussian distribution, they made sure prices cannot drop below zero.

(Remember? We adopted a similar approach in the previous article to train our Gaussian linear regression model by asking it to predict log-variance instead of variance. These patterns keep surfacing in totally different situations.)

While switching from a Normal to a Lognormal price distribution offers the added advantage of a slightly heavier tail, it still fundamentally fails under market stress. This is because a lognormal price model structurally assumes that percentage market returns are still normally distributed.

To accurately model a market capable of swallowing a bank like Lehman Brothers overnight, financial risk systems must pivot toward true power-law, fat-tailed distributions, such as the Student’s t-distribution. In these models, extreme anomalies are expected parts of the system rather than statistical anomalies.

Embracing Fatter Tails

The core intuition behind the Student’s t-distribution lies in its flexibility. While a Gaussian curve is rigidly defined by just its mean and variance, the Student’s t has three parameters: the peak, the width and nu (ν). The tail’s thickness is determined by nu which represents the degrees of freedom.

The Student in question was William Gosset. He was using Statistics for the noble purpose of making better beer. Since he was forbidden to use his name while publishing, he signed his work under the pseudonym “Student”.

When the degrees of freedom are low (e.g., nu = 3), the distribution stretches out. The peak squashes down, and the tails become significantly fatter. This mathematically assigns a much higher probability to extreme, “impossible” outliers.

As the degrees of freedom approach infinity, the Student’s t-distribution seamlessly morphs back into a standard Gaussian curve.

By switching to a Student’s t-distribution, risk managers stop treating 2008-scale meltdowns as once-in-a-universe anomalies. Instead, the math accepts them as an inevitable cost of doing business in a connected, herd-driven world.

Implications in Probabilistic AI

As we have seen, defaulting to a standard Gaussian framework in deep learning, manifests as the Mean Squared Error (MSE) loss function which many of us frequently use.

Because the Gaussian distribution assumes extreme outliers are virtually impossible, its mathematical byproduct, our favorite MSE loss, penalizes errors quadratically… much more than it ideally should. An outlier far from the mean creates a huge penalty (due to squaring of the error in MSE loss). This causes the “gradients” to explode during training, dragging the entire neural network’s weights toward predicting the outlier and destroying generalization. In short: MSE allows a tiny handful of outliers to completely hijack the entire learning process.

To combat this, modern robust deep learning increasingly relies on the Student’s t Loss Function.

The Student’s t Loss Function (Robust Regression)

By replacing the Gaussian likelihood with a Student’s t-distribution, the loss scales logarithmically with respect to squared residuals rather than quadratically at extreme ranges.

PyTorch provides a built-in implementation via from torch.distributions import StudentT. You can instantiate the distribution directly by passing your model's prediction to the location parameter (loc). Computing the robust loss simply requires evaluating the negative log-probability density of the target labels:

import torch
from torch.distributions import StudentT

# Regular logic to calculate your model's predictions comes here

# 1. Instantiate the distribution with the model's prediction as the location
# df = degrees of freedom. 2.0 is a good fat tail.
# scale parameter controls the spread of the distribution (like std deviation) 
# but has a slightly different formula.
dist = StudentT(df=2.0, loc=prediction, scale=1.0)

# 2. Compute the batch mean Negative Log-Likelihood (NLL) loss instead of MSE
loss = -dist.log_prob(target).mean()

# Use this loss in your training loop

Note that when df is at 2 or below, the variance of the distribution becomes infinite. You get an idea now why the gradients stop exploding: a distribution that already expects outliers barely raises its log-density when one arrives, and it is that log-density your gradients come from. You cannot shock a system that never expected its data to behave nicely in the first place!

The utility of fat-tailed math extends far beyond basic robust loss functions. It is actively solving core bottlenecks in predictive confidence and edge-device hardware efficiency.

Fat tailed Math in Autonomous Systems

A shift to Student’s t-distributions is becoming the gold standard in autonomous systems. For example, networks like T-DistNet explicitly use Mixtures of Student’s t-distributions to predict vehicle trajectories. Why? Because standard Gaussian models have ‘thin tails’ that treat rare, dangerous driving outliers as impossible 25-sigma events and may fail to flag them. By adopting the Student’s t (whether via evidential priors or direct mixture modeling), self-driving cars can account for real-world chaos and navigate highly unpredictable driving hazards.

LLM Quantisation (Student Float / SF4) for edge computing

Recent research in Large Language Models reveals that the weights of massive networks (like LLaMA) do not follow a normal distribution; they exhibit heavy-tailed behaviours! Keeping this behaviour in mind when writing quantization (model compression) algorithms help.

• Traditionally, compression algorithms like Normal Float (NF4) are optimized under a Gaussian assumption. They allocate quantization bins based on the probability mass of a bell curve. But real LLM weights have genuine outliers which contain vital information.
• Forcing these heavy-tailed weights into a Gaussian quantization framework creates a dilemma: you must either aggressively clip the outliers (destroying signal) or scale the entire range to accommodate them (wasting precision on the vast majority of central values).
• To address this, AI researchers introduced Student Float (SF4), a specialized 4-bit data quantization format based entirely on the Student’s t-distribution. This format allocates appropriate precision to the heavy tails without sacrificing resolution in the center. The result is high-fidelity LLM inference on edge devices where every bit of compression counts.

So you see, William Gosset’s century-old beer math has successfully, er, brewed its way into modern neural networks, ensuring today’s LLMs are resilient enough to handle real-world chaos.

There are many more such fascinating distributions out there, each with its own distinct shape, parameters and colorful history. But we will take a pause here and move on to our next major topic: What happens when we mix these distributions?

Modeling Multivariate Random Variables in ML

Often, real life involves an interplay of multiple Random Variables. The focus shifts from individual random variables to the broader “multi-variate distributions” behind them. We can do a lot of AI magic by understanding how distributions work when combined! While the underlying math can get intricate, our aim in this article is to capture enough “gist” to harness these concepts practically.

For example, our health is a result of an interplay between individual random variables involving genetics, diet, lifestyle, and maybe our stars! Every variable involved has its own distinct distribution. The probability density function (PDF) of our overall health is a joint distribution of all hundreds of variables involved. Yes, next time someone asks me for my horoscope chart, I promise not to look quite as skeptical.

If the constituent random variables are independent, the resulting distribution is relatively easy to calculate. But in real life, the constituent random variables are usually dependent on each other. We need to invent a metric to measure this “dependency” and factor that in, to get our final probability estimates.

The impossible odds of being Arnold Schwarzenegger

I have always admired Arnold Schwarzenegger. He was the topmost bodybuilder, became a massive Hollywood star, launched highly successful businesses, and authored bestselling books (his New Encyclopedia of Modern Bodybuilding is a classic and highly sought after by amateurs and professionals alike). Then, he became a successful politician, serving as the Governor of California twice. As of 2026, at age 79, he is the undisputed king of fitness on social media, leading Instagram’s fitness creator category.

Each of these achievements independently carried staggering odds. If you multiply the standalone probability of each event, the resulting probability is so microscopically low that replicating his success seems mathematically impossible. He just shouldn’t have happened.

But are we right to multiply these individual probabilities?

Of course not. These events heavily influence one another. Once Arnold became a world-class bodybuilder, the doors of Hollywood became much more accessible to him than to a random person on the street. Instead of a one-in-a-million shot at Hollywood stardom, his probability jumped significantly (say to 1 in a 100). The same pattern of path-dependency applies to his business and political careers. Acknowledging this conditional probability doesn’t diminish his incredible drive; rather, it explains how real-world dependencies slash the odds from one-in-a-trillion to something realistic.

While this dependency feels obvious in hindsight, ignoring it can have catastrophic, real-world consequences!

Take the tragic case of Sally Clark. When her second child also died due to sudden infant death syndrome (SIDS), she was wrongfully convicted of murdering her two children. What was the evidence? An “expert” convinced the court that the odds of two SIDS deaths occurring naturally in one family were 1 in 73 million. He reached this number simply by squaring the odds of a single SIDS death which was 1 in 8543.

This logic was deeply flawed due to multiple reasons. For one, the deaths were treated as independent events, completely ignoring shared genetic and environmental factors. In reality, if a family loses one child to SIDS, the probability of a second occurrence increases dramatically. Thankfully, the Royal Statistical Society intervened, exposed this flawed assumption of independence, and sought her release.

Moving from Events to Variables: Covariance

The takeaway is clear: just like with individual events, we must resist the temptation to assume independence when multiple random variables are at play. To build accurate models, we need a definitive metric to measure how these variables move together. This metric is Covariance.

Imagine a system with 10 different random variables. Instead of tracking their interactions in isolation, we can neatly organize their relationships into a 10x10 square Covariance Matrix.

• Every column-header and row-header in this matrix represents a random variable, and the value in each (i, j) position is the Covariance between the i’th and the j’th random variable.
• The diagonal of this matrix is the set of 10 individual variances of these random variables.
• Moreover the matrix is symmetric, so technically you will have 45 unique covariance values.

Two limitations to be kept in mind:

• It only sees lines: By definition, Covariance measures linear relations only. So it is possible for it to be 0 and yet the variables be related (literally, in some crooked way).
• It is scale dependent: A raw covariance value does not tell you the strength of a dependency because it is bound to the scale of the data. To fix this, we can use Correlation which normalizes the covariance by dividing it by the product of the individual standard deviations, making this metric fall into a clean, interpretable -1 to +1 scale.

From Measuring Dependencies to Building Models

To predict complex outcomes like the weather, an aircraft’s trajectory, or a financial market, our models must combine multiple dependent or independent distributions into a unified system.

Get Allohvk’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

There are many ways to mix and model these distributions. Let us look at a couple of them.

Summing Variables: The Arithmetic of Convolution

Let X and Y be dice throws. Each has a Probability Distribution of 6 outcomes with 1/6 probability each. Let Z = X+Y. Then Z is also a random variable and has possible outcomes from 2 to 12. Can you guess Z’s Probability distribution?

This is not as simple as adding the probabilities. In fact, X and Y were uniform distributions but Z is not! The number 7 is more likely to turn up than say 12 or 2. This is because many combos leading to 7 like 1;6, 2;5, 6;1 etc whereas there is only one combo giving 2 which is 1;1. So Z has its own probability distribution and is no longer uniform. Its shape is determined by an operation called convolution of the original distributions.

Let us go back to our favorite distribution... yes, the Gaussian. This distribution possesses a remarkably elegant property: it is closed under addition. This simply means: If X and Y are Gaussian random variables, their sum S = X + Y is also guaranteed to be a Gaussian random variable.

This remains true even if they are heavily dependent on each other, provided they are jointly Gaussian (i.e. the distribution of the joint (X,Y) is Gaussian.)

Practical Application — Modern Portfolio Theory: This neat property is the foundational bedrock of modern finance. Imagine your investment portfolio consists of two assets: a tech giant X specializing in online search and a grocery chain Y renowned for its ability to give great bargains to its consumers. Your daily portfolio return is the (weighted) sum of their individual performances: S = X + Y.

Under standard financial models, daily stock returns are approximated as Gaussian distributions. Because the sum of Gaussians is also a Gaussian, a portfolio manager does not need to run slow, complex computer simulations to understand your total financial risk). Instead, they use a much simpler formula:

If the tech stock and the grocery stock move completely independently, their covariance is zero, and you simply add their variances. If they influence each other, that exact interaction is safely captured by the covariance term.

By scaling this system up to a multivariate network using linear algebra, a modern risk engine can evaluate the exact parametric Value-at-Risk (VaR) of a portfolio containing dozens of assets in microseconds.

Note: While this Gaussian assumption enables lightning-fast matrix calculations, it introduces back the fat tail-risk we discussed earlier and should not be relied upon in isolation.

Of course, as we saw earlier, adding (lots of) random variables together usually makes things more Gaussian because of the Central Limit Theorem. But what if we want to make things less Gaussian?

Mixture Models (example GMMs)

We know the Gaussian bell curve by now. It is unimodal, i.e., it has a single peak. What if you wanted to model a data-generating phenomena whose PDF has several peaks? These are called multimodal distributions.

Consider the simple case of plotting the heights of every adult in a city. You may assume this results in a bimodal shape: a PDF with two distinct peaks (one centering around the average female height, the other around the average male height) but, in reality, the difference in male and female heights is minor, relative to their individual variances, and the bimodal shape is not apparent. Now consider a mixture of professional basketball players and the general female population. This time, the gap stretches wide enough that a second peak visibly emerges. Because a standard, single Gaussian lacks the flexibility to model this “camel hump” shape, it fails completely. What do we do?

To solve this, we don’t add the random variables; we mix them.

In a Mixture Model, we assume the data is generated by selecting from one of several “constituent” distributions based on a probability weight. A common example is the Gaussian Mixture Model (GMM), where the data is assumed to be generated from a mixture of multiple Gaussian distributions. See fig below.

Take the simple case of just two constituent Gaussian distributions, say X = Gaussian(mean=0,var=1) and Y = Gaussian (2,3). Let the final samples come from the first Gaussian 90% of the time and from the second Gaussian, the remaining 10%. That is it. We have a mixture of distributions. If we take 100 samples from this mixture, we are likely to see 90 samples belonging to the first Gaussian X and 10 samples to the second Y.

An alternate way to think about it is that we effectively flip a weighted coin for every data point of Z. If the coin lands heads (90%), we draw the sample from X. If tails (10%), we draw from Y. The resulting combined shape is not a Gaussian. It is a complex, double-humped terrain, which is exactly what we wanted to model.

Introducing the latent variable: But here is the catch: when we look at real-world data, we only see the final values. We don’t see the coin flip. We don’t know which distribution a specific data point came from. That “coin flip” is hidden from us.

This brings us to one of the most important concepts in Probabilistic ML: The Latent Variable. You will see it being used in many situations.

As the name suggests, “latent” variables cannot be directly measured in our raw data. But they silently influence the variables we can observe.

In case of a GMM, a latent variable L ∈ {1, . ., K}, which specifies: which of the K distributions generated the output.

We saw an example above where two Gaussians were mixed i.e. K=2. In that case, L ∈ {1,2} and with a 90% probability that it will turn out to be 1 and 10% probability that it would be 2. The mixture Weights were 0.9 and 0.1. These are literally just the probabilities of this hidden switcher! They act like Lego blocks, allowing us to combine simple Gaussians to map incredibly complex, multi-peaked data terrains.

Naturally, the parameters of a GMM model are no longer only the means and the covariance matrix of the constituent Gaussians. We also need to include the mixture weights of the components.

In theory, we could mix and match any kind of distributions. But in practice, we usually see parametric mixture models, where the distributions are all from the same family, but with different parameters. For instance they might all be Gaussians with different mean and variances.

From Distributions to Models: Mixture-of-Experts (MoE)

We can take this logic one powerful step further. Instead of just mixing simple probability distributions, what if we mixed entire models?

Imagine mixing a Regression model that is great at predicting low prices with another that is great at predicting high prices. Here, too, you can have a latent variable L, controlling which model to use.

This is called a Mixture-of-Experts (MoE) model. In this architecture, each predictive model is an “expert,” and a “Gating Network” determines which expert to use when.

Noticed a key difference between a GMM and MoE? In a standard GMM, the weights are static (e.g., always 90/10). In an MoE, the weights are conditional and dynamic. The gating network evaluates the incoming feature vector X in real time routing text token-by-token in LLMs and dynamically scales the contribution of each expert. The final output is not a hard switch, but a softly weighted combination of the experts’ predictions.

This evolution from static mixtures to dynamic MoE routing is what allows modern AI to scale. While we study MoE in later articles, let us end this one by looking at a practical application of GMM.

Practical Application: Gaussian Mixture Models (GMM) vs. K-Means Clustering

In the previous section, we saw how to construct a GMM. What if we mastered the art of reversing this process? Can you think of any area where this might help?

Clustering is an ML technique that groups a bunch of data points into distinct categories based on similarities. For e.g., in retail, it is used to group users with similar buying habits for targeted marketing or personalized recommendations. In the field of security, it is used to find anomalies in a given data set (say a collection of user logs). K-Means is a great clustering algorithm. But in many situations, GMM is better.

Imagine a dataset with K clusters of data. It would not be far-fetched to assume that data in each cluster comes from a different distribution. The entire dataset can be viewed as a mixture model, with each data point coming from one of K constituent distribution with some probability. So instead of doing all that “centroid business” as done in K-means, how about simply separating datapoints based on the underlying Probability distributions?

Isn’t that an elegant way to cluster a dataset?

Here’s how we go about it. Let’s assume the dataset is a GMM i.e. a mixture of K Gaussians. As we established, a latent variable decides which distribution to sample from and can take values from 1 to K. Let us call this latent variable Z, which is the usual convention.

We need the value of Z for every single data point in the dataset. Let us start with a global set of weights (w1,w2….wk) that sum to 1, representing the probability of choosing each of the K clusters. There is a name for this kind of weights — a Categorical distribution. It is just a formal sounding name for a bunch of weights that sum to 1. These weights act as a mathematical placeholder, representing the prior probability of each cluster before we look at any data points.

Modeling a Simple GMM Scenario

Let us take a simple case where the dataset comes from a mixture of 3 simple Gaussians i.e. K=3. Each Gaussian is parameterized by a θ which is a mean/variance combo. Our mission is to find the parameters of this GMM. This is a combination of the individual parameters θ1, θ2, θ3. Apart from this it also includes the weights w1, w2, w3. If the three global weights are distributed as [0.8, 0.15, 0.05], this means the dataset will have approx 80% samples coming from Gaussian1, 15% from Gaussian2 and 5% from Gaussian3.

We don’t know any of these six quantities. However, we do have the observed data. This creates a classic chicken-and-egg problem: if we knew which data points belonged to which Gaussian (Z), we could easily estimate the parameters. Conversely, if we knew the parameters, we could easily deduce which Gaussian each data point came from.

But how do we go about it?

Solving GMMs Using the Expectation-Maximization (EM) Algorithm

To solve this, we use something called the Expectation Maximization which is an iterative algorithm.

1. Expectation (The Guess or the E-step): We start by guessing initial parameters for our Gaussians (θ₁, θ₂, θ₃) and our global weights (w₁, w₂, w₃). Using these, we calculate the individual probabilities (the “responsibility”) that each data point belongs to Cluster A, or B, or C. There is a simple formula for this which we need not get into here. Note that this needs to be done for each data point.
2. Maximization (The Update or the M-step): At this point, we have all the responsibilities i.e. for each data point, we know the probability of it belonging to Cluster A, B, or C. We can now re-estimate the best possible (w₁, w₂, w₃) and (θ₁, θ₂, θ₃) using weighted MLE. Wecovered MLE in article 4.

Using this new parameter-set, we re-do the E-step and get a new set of responsibilities. Then re-estimate the GMM parameters with MLE again. Repeat until convergence. See EM in action here.

Why GMM Outperforms K-Means on Complex Datasets:

Why go through all this trouble? Because K-Means has two major flaws that GMMs fix:

• Soft Clustering: Unlike K-means, which assigns each point to a single cluster, in GMM we get soft clusters: i.e. for each datapoint, we get a set of responsibility scores showing how likely that point is to belong to each cluster. A probabilistic cluster assignment is more flexible for ambiguous data points sitting between clusters.
• Accounting for Variance: K-Means only measures circular distances from a center, forcing clusters into rigid spheres. MLE helps find θ which includes the variance. Since, K-Means does not account for variance, it performs poorly for non-spherical shapes. See fig above how regular clustering considers only circular distances from the centre whereas GMMs can models clusters with various shapes and orientations.

The code is simple and straightforward due to readily available libraries:

from sklearn.mixture import GaussianMixture

# Assume 'X' is your real training dataset. n_components is K (num of clusters)
# random_state is optional. It simply ensures you get the exact same clusters 
# each time you run your code. Else there could be mild differences due to 
# the probabilistic nature of the algorithm
gmm = GaussianMixture(n_components=3, random_state=42)

# 2. Run the EM algorithm loop to learn the parameters (w and θ)
gmm.fit(X)

# 3. Perform soft clustering to get responsibilities for each data point
responsibilities = gmm.predict_proba(X)

# For hard clustering, use ".predict" to directly get the assigned cluster
# for each data point
labels = gmm.predict(X)

GMMs in the Wild: Other Practical Implementations

Let us look at a few interesting uses.

• Computer Vision: If you have ever seen an automated traffic camera track vehicles or a security feed highlight a moving intruder, you have seen a GMM at work. In video analytics, the “background” isn’t static; trees sway, shadows shift, and lights flicker. A simple frame-by-frame subtraction fails completely. Instead, algorithms like the Stauffer-Grimson method model the history of every single pixel as a mixture of Gaussians. The system learns to classify steady pavement and swaying branches as high-probability background modes, while identifying a passing vehicle as a transient, low-probability outlier.
• Kaggle “Instant Gratification” Competition: GMMs were the secret weapon used by the winners. The dataset consisted of highly complex clusters. While standard clustering algorithms failed, top competitors realized the data was generated by a mixture of Gaussians. By fitting GMMs and QDAs to the data, they effectively “reverse-engineered” the generation process. Instead of trying to predict the labels directly, they modeled the shape of the clusters, allowing them to define the decision boundaries with near-perfect mathematical precision. See code and detailed explanation here.
• The “Original” Generative Model: Long before GANs or Diffusion models dominated AI headlines, GMMs were the original generative algorithms. Because a GMM maps the full PDF of a dataset, it doesn’t just cluster data, it can synthesize it. Today, this makes GMMs a lightweight powerhouse for Privacy-Enhancing Technologies (PETs). By training a GMM on sensitive medical or financial records, researchers can sample entirely new, synthetic data points that mimic the exact statistical correlations of real patients or transactions, without exposing a single byte of actual private data.

We started the article by looking at distributions other than the Gaussian. We saw how the Student’s t-distribution holds great relevance in modern AI.

We then mastered the art of combining random variables. We saw how Covariance measures the hidden influence they have on one another. We summed them up via Convolution to measure the health of our financial portfolio and blended them via Mixture Models to map complex, multi-modal data. We then reversed that process to discover a neat clustering technique that outperforms K-Means.

We now know how to build the "whole" from the parts.

But in data science, the challenge is often the reverse. When presented with a complex system, how do we slice it open? How do we isolate one specific variable while holding the others constant? How do we update our beliefs about the whole system when we only see a tiny fragment of it?

In the next article, we will revisit the PDF landscape and master the holy trinity of Bayesian statistics: Joint, Conditional, and Marginal Distributions. These terms sound dry but to bring these concepts to life, we will leave behind the world of bodybuilders and bankers and instead, step into the private studios of Leonardo Da Vinci and sit across the board with Chess Grandmasters.

Onto the art and the games...

This post is actually a live draft of a chapter from a book on Probabilistic ML I’ve been writing. In a couple of weeks, I’ll be launching it and I’m giving away the first 10 chapters completely free to early readers. If this intuitive approach to complex math makes sense to you, let’s stay connected here.

Other articles in this series:

1. The Bomber, The Brinjal, and The Bot: A Bayesian Tale
2. Probabilistic AI and the Art of Expressing Ignorance: From Overconfidence to Computational Humility
3. Demystifying Random Variables, the Engine of Uncertainty: Making sense of a Chaotic, Continuous World
4. The Emergent Curve — Stepping Beyond the Illusion of Point Estimates: An intuitive, fun journey through the bell curve, Gaussian regression, and the path to true probabilistic AI
5. The Road Less Travelled — Venturing beyond the Normal: Intuition Behind Fat Tails, Student’s t, Mixture Models, and GMMs