And you realize the answer isn’t obvious. Precision went up by 3 points, but on how many images? Recall dropped on one rare category; is that real, or just noise? The classifier looks great on your test set, but will it hold up on real-world data?

I’m Oscar Bouvier, research engineer at Heuritech. At Heuritech, we analyze millions of social media images to detect over 2,000 fashion attributes, and turn that into trend forecasts for fashion and sportswear brands. Our Data Science team trains specialized deep-learning models, one per garment family (dresses, pants, shoes, tops…), each detecting hundreds of fine-grained attributes.

And a model is never uniformly good or bad; its quality varies sharply from one attribute to the next, and so does the amount of data used to assess it. “Bleach Denim” fabric might be rock-solid while “Poet” sleeve is a coin flip. The labels that matter most are the ones we see least, and so the ones we can judge with least confidence: a niche detail meaningful to a client, or an emerging trend still on the rise.

Alongside us, the Ops Team safeguards the quality and completeness of what reaches the platform: they turn client demands into concrete attributes, collect the relevant data, and orchestrate the retrains.

After every retrain, both teams face the same recurring question: for any single attribute, is the model good enough to ship?

This article tries to answer this question, resting on one central idea: a metric isn’t a model’s true behavior but an observation of it, and every observation carries a degree of confidence.

Bayesian probability is the practical framework that quantifies that uncertainty and turns it into diagnoses and actionable decisions, giving the Ops Team what it needs to make the right move.

The mathematical backbone of the approach comes from a 2005 paper by Goutte and Gaussier, “A Probabilistic Interpretation of Precision, Recall and F-score”. It’s an elegant, well-established result that we’ve found genuinely useful in production. We’d like to see it become a standard tool, and this article is our case for adopting it!

Everything here is available as reusable Python code in this repo, with examples, plots, and animations illustrating the different practical use cases. The bayesian-precision-recall package is also available on PyPI. Check it out!

pip install bayesian-precision-recall

GitHub - OscarBouv/bayesian-precision-recall: Bayesian precision-recall model for ML evaluation with production drift detection

If you want the bigger picture of the team’s research, and our efforts to push model performance and trend-insight quality as far as possible, take a look at my colleagues’ articles!

• Étienne on designing the cutting-edge forecasting models that power Heuritech’s trend insights: Is Fashion Predictable? The Rise of Foundation Models
• Paul on our deep-learning vision stack: A social media image is worth a thousand fashion labels…
• Louis on our systematic approach to tuning every model for the best possible performance: Stop Guessing: A Systematic Methodology for Tuning Deep Learning Models

Part 1: The two questions every attribute has to answer

Every data science student learns precision and recall early on, usually as a pair of formulas to memorize. But it’s worth pausing on what they actually mean.

When a model looks at an image and declares “this is a bleach denim jacket,” there are two different ways it can be wrong, and they map onto two metrics.

Precision answers: When the model says “bleach denim jacket,” how often is it right? A high-precision detector is rarely wrong. When it makes a claim, you can trust it.

Recall answers: Of all the bleach denim jackets that were actually there, how many did it find? A high-recall one rarely misses anything, even if it occasionally over-claims.

These two pull in opposite directions, and either is easy to game at the other’s expense. The extreme cases make it clear:

• Predict only what you’re sure of (~near 100% certainty threshold) : precision is perfect, recall can be near zero;
• Predict every image (~0% certainty threshold): recall is perfect while precision is near zero;

Alone, each is meaningless, which is why the F1-score rolls them into a single number. For us, though, they don’t carry equal weight, and the reason shapes everything below.

Part 2: Why precision matters most for us

What Heuritech sells is trend insights. The question for a client is less “is there a trench coat in this photo” but more “is the trench coat growing or shrinking, in which market, and how fast.” This is driven by the relative volume of an attribute over time (its trajectory), and a trajectory is only trustworthy if the underlying counts are clean and stable.

The two metrics matter for different reasons: precision keeps the trajectory’s shape honest, stopping false detections from bending it, while recall keeps its volume high enough that the signal stays statistically meaningful. Steady errors are survivable: a fixed fraction of detections are wrong, or a fixed fraction missed, just scales the counts by a constant factor and doesn’t impact the shape. The figure below tells the story for false positives.

The problem is false positives whose rate shifts with season, platform, or geography, such as a new Instagram filter that fools the classifier, a regional styling it never saw, a seasonal look-alike garment. Any of these can make false positives, so as the underlying trend, surge while the real one is flat. High precision ensures that drift is small enough that the shape stays trustworthy.

Recall can drift too: if the model’s miss rate increases over time, a flat trend can show up as a decline.

What tips the balance is the direction of error: precision drift tends to invent or inflate a trend, while recall drift mostly dampens a real one, and reporting a rise that isn’t happening is the failure we least want to hand a client.

That’s why we set asymmetric minimum bars for shipping an attribute, consistently favoring precision over recall, reflecting our preference to miss some real garments rather than distort the growth signals our clients act on.

The target thresholds are set granularly depending on the criticality of the attribute, its business impact, and its visual distinctiveness.

Precision keeps the signal honest; recall keeps it readable and we need every attribute to clear its respective bars.

Part 3: A metric is an estimate, not a fact

We never actually know an attribute’s true precision. What we call “precision = 80%” only ever means “out of the 100 images we happened to check, 80 were correct.” The true precision, the rate at which the model would be right across all possible images, is invisible. We estimate it only through a finite, often small, sometimes biased sample.

Take a coin you suspect is unfair. Flip it 10 times and get 7 heads (70% rate). We can’t tell yet if the coin is biased: 7 out of 10 can happen with a perfectly fair coin, so 50% is still entirely plausible. Now flip it 1,000 times and get 508 heads, 50.8%. The observed rate is closer to even, but more importantly it is now much more reliable: a thousand flips leave almost no room for the true rate to be far from 50%. Confidence comes from the volume of flips behind a rate, not from the rate itself.

Estimating an attribute’s precision is exactly the same problem, except that heads and tails become true positives and false positives.

Part 4: From a single number to a curve

This is where Bayesian modeling enters. Instead of reporting a single number, we report a distribution: a description of which true precision values are plausible given the data, and by how much. Bayesian inference is exactly the machinery that turns a raw count of true and false positives into a probability distribution, represented by a curve. For precision and recall, that curve takes the form of a Beta distribution, a flexible family that lives neatly between 0 and 1, exactly the range a probability can take.

For a deeper look at the mathematics behind those concepts and further explanations on Precision/Recall, see the animations in the final section!

More data gives a narrower curve and more confidence. A wide, flat curve means “the true precision could be anywhere.” A tall, narrow spike means “we’re quite sure.” Through all of it the curve stays centered on the precision we actually observed; our best single guess remains the raw measured rate while our confidence in it grows.

Part 5: Two real attributes from our data

For the rest of this article we’ll draw our entity examples from one classifier, the pants attribute model (loose jeans, cargo pocket, equestrian pants, etc.). Here are two of its labels from our actual pipeline. Loose jeans: 80 true positives, 26 false positives. With over a hundred predictions behind it, the curve is tight. Equestrian pants: just 5 true positives, 6 false positives. With only 11 predictions, the curve is very wide and the real performance remains uncertain.

This is why we never compare bare numbers. The loose-jeans 75% precision is something we can act on; the equestrian-pants 45% could just as easily be 30% or 60%, the same kind of number but a completely different decision.

A useful quirk: for a fixed number of samples, extreme estimates come with more confidence than middling ones. A model that nails 10/10 is more convincingly good than one scoring 6/10 is convincingly average.

Part 6: The diagnosis: the three gates every attribute must pass

A metric estimate alone is not enough to decide whether an attribute is ready. This is where real-world industrial constraints comes into play.

We ask three questions:

• does it clear our quality bars?
• is it better than the model currently in production?
• will it hold up on real-world data?

The same Bayesian framework can answer all three by measuring confidence, not just performance.

Question 1: Does it confidently clear the bar?

Using the posterior curve and our in-lab testing data, we compute the probability that the attribute’s true precision and true recall exceeds their respective floor. What we care about is how much of the curve sits above the bar, not merely where its peak lands. In practice, this probability is simply the area under the posterior curve that falls above the threshold (the green region).

An attribute is considered “satisfying” only when that probability is high (>80%) and “insufficient” when it’s low (<20%). We flag “borderline” performance when confidence is close to the acceptance threshold (70–80%), since a small amount of additional data could change the verdict. Anything in between is “uncertain”.

Question 2: Is it better than the model currently in production?

We also ask whether the new candidate actually improves on the model currently serving it. Instead of comparing two point estimates, we compare two curves, asking for the probability that the new model is genuinely better: P(p_new > p_old). Since the difference of two Betas has no tidy closed form, we estimate it by sampling from each curve and counting how often the new model wins. See how this sampling strategy can actually converge to a precise estimate.

We can now turn these probabilities into actionable verdicts: Certain gain (P > 0.80), Certain drop (P < 0.20), Stagnation (in between).

A certain drop matters: the retrain has made this attribute worse than what’s already live, and needs an investigation. A stagnation means the change was real but neutral.

Question 3: Will it survive real-world data?

A classifier can clear the bar and beat the old model on the test set, then fall apart on production data, where new filters, regional styles, and shifting habits can differ. This is domain shift, and on social media it’s the default.

The same Bayesian tools handle it: we look at the posterior of the difference between the two rates and ask whether it’s big enough to matter, using a small Region of Practical Equivalence (ROPE) around zero. If the credible interval sits inside the ROPE the metric transfers; if it sits clearly outside, the drop is real; if it straddles the edge, there isn’t enough data to tell.

Two things matter at once here, how big the gap is and how much data supports it. A big drop on lots of data is a five-alarm fire. A big drop on five images is a shrug.

Part 7: From diagnosis to action with the Ops playbook

The shipping decision belongs to our Ops Team, the people who safeguard the quality and completeness of our platform insights. For every attribute they decide the next step. Is it already good to go? What kind of data should we target? This framework is what gives them the tools to choose with confidence.

Each failure mode points to a different remedy, and each remedy becomes a concrete task for our Ops team. Rather than label it “insufficient” and move on, we route it.

The five situations we route on, and their responses:

• Not enough data (wide, uncertain curve). The estimate is too uncertain to trust either way. Ops launches a targeted acquisition: rather than tagging at random, we run a text embedding-based search that queries our image pool with the attribute’s description and combine it with active learning that prioritizes the cases the model is least sure about. The goal isn’t more data in general, but more data for this specific class, found efficiently.
• Bad production transfer (test and prod curves separate). The attribute works in the lab but not in the wild. Ops runs a precision audit on true production positives, pulling the actual predicted-positive images from prod and hand-checking them to find the failure mode (a new filter? a confusable look-alike garment? a regional styling we never trained on?). The audit tells us what kind of data to go collect.
• A certain drop against the live model. An existing attribute comes back confidently worse than what’s shipped, usually because a newly added class confounds an old one (“cargo pocket” bleeding into “patch pocket”). Ops finds the offending pair and either tags more examples along the confusion frontier to sharpen the boundary, or redefines the class architecture if the two simply aren’t separable.
• Borderline precision or recall. The estimate sits near a bar but the posterior curve still spills across it. If it’s precision, the model is over-claiming on a confusable category, so Ops mines hard negatives to push that curve above the floor. If it’s recall, the fix points the other way: tag more positives, especially the harder ones the model misses.
• Far from any threshold (clearly above or clearly below). Not every verdict needs a retrain. If a class sits comfortably above both floors with a tight curve, the right action is to ship it now and move the team’s effort elsewhere. If it sits far below, and acquisition has already had its chances, the honest call is to deprioritize it, or redefine the attribute into something the model can actually separate. Knowing when not to spend another cycle is as valuable as knowing when to.

Each remedy feeds a retrain until every attribute clears the bars. For a critical new attribute, one pass is rarely enough.

Shipping a brand-new critical attribute reliably can take up to four retrain-and-acquisition cycles, each loop sharpening a different weakness (first volume, then transfer, then the stubborn confusions) until it clears all three gates with enough confidence to trust.

The discipline of the loop is what lets us promise clients trustworthy insight, not just available insight.

Why this changes how we work

The change is easy to state and far-reaching in practice. We stopped treating per-attribute metrics as facts and began treating them as beliefs with a measurable degree of confidence. It helps us avoid overreacting to sparse data, detect meaningful regressions earlier, and direct data-collection efforts toward the attributes where additional evidence will have the greatest impact.

More importantly, it sets the standard an entity must meet before reaching production. Instead of relying on an uncertain performance estimate, we require enough evidence to be confident that the result is real. The goal is not just better metrics, but more trustworthy ones.

That discipline is what lets the trends we report stand on solid ground, so that when our platform tells a brand a dress silhouette is rising in South Korea, the number reflects what is actually happening, not what a thin or shifting sample merely suggested.

Beyond Heuritech: build your own decisions

The three use cases presented here (acceptance thresholds, model-to-model comparisons, and production drift detection) are simply the questions we needed to answer. They are not special cases of the framework, they are consequences of it.

Once precision and recall are modeled as probability distributions, uncertainty becomes something you can reason about. From there, designing new decision rules is surprisingly easy. The question is no longer “what is the metric?” but “what is the probability that the metric satisfies the condition I care about?”.

If you’re a data scientist or ML engineer, and this kind of problem shows up in your own work, I encourage you to give it a try! The Bayesian machinery stays the same; only the question changes.

⤷ Bonus : the maths, for the curious

Let’s get a bit deeper into the bayesian precision and recall modelling.

Take a very simple prediction task, like predicting whether the animal in an image is a cat or a dog. The “cat” true precision is an unknown number p ∈ [0,1]. We never see p directly; we only see counts of true positive (right “cat” detections) and false positives (predicted “‘cat” when it’s actually a “dog”) named TP and FP. The same goes for recall r ∈ [0,1] with counting false negatives (missed “cat” predicting “dog”).

Each predicted-positive “cat” image is then one biased flip trial : given unknow true precision is p, that prediction is correct (TP) with probability p and wrong with probability 1-p (FP).

Similarly each labeled-positive “cat” image is predicted with probability r (TP) and missed with probability 1-r (FN).

The link between the two is the likelihood: the probability of seeing those counts, assuming the true precision is some value p, that follows Binomial distribution (same as biased coin flips). Using a well-chosen Beta prior , the posterior is also a Beta distribution.

The uninformative prior applies when choosing Beta(1, 1) (α=β=1), collapsing to a uniform (flat) prior.

Have a play with this Desmos visualization to see how the prior, likelihood, and posterior interact, and how we turn the posterior into the probability of clearing a minimum acceptance rate!

Bayesian Precision Posterior

Recall is then the mirror image. Precision asked, “among the images we predicted positive, how many were right?”, conditioning on the TP + FP pool. Recall asks, “among the images that truly are positive, how many did we catch?”, conditioning instead on the TP + FN pool. The posterior then follows naturally by the same Beta-conjugacy argument:

So both metrics share the same machinery, a Beta posterior driven by two counts, differing only in which pool the true positives are measured against: predicted-positives (TP+FP) for precision, actual-positives (TP+FN) for recall.

Should We Ship This Model? A Bayesian look at precision and recall, and how to turn it into actions was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

You’ve run 47 hyperparameter sweeps. Each one takes 6 hours on 4 GPUs. Your validation metric has improved by 0.4% in two weeks, and it may have been a lucky run.

Your experiment tracker has a graveyard of rejected ideas:

• Cosine annealing: no improvement.
• Weight decay: hurts performance.
• Muon: unstable.

But you tested cosine annealing with a learning rate tuned for a constant schedule. You tested weight decay on SGD but never tried it with Adam. You tested Muon with Adam’s learning rate range. None of those ideas were bad. The method was, and most of them probably would have worked given a fair shot.

This article shows you how to give them one. You’ll learn how to separate the parameters you’re studying from the ones you need to control, how to explore search spaces efficiently with quasi-random sampling, and how to sequence your experiments so each round builds on the last instead of invalidating it.

This methodology draws on my years of experience — directing research teams at 42 AI’s research lab, and teaching at Paris 1 Sorbonne and Amsterdam Tech University — as well as sources I’ll link throughout, particularly the Google Deep Learning Tuning Playbook [1] and Goodfellow et al.’s debugging strategies [1b].

One piece of context before we dive in: this work was part of migrating our production image classifiers from TensorFlow to PyTorch. Our TensorFlow baseline was a formidable target: it encodes a decade of accumulated knowledge from researchers and engineers, some of whom are now at Mistral and DeepMind. So “beating the baseline” meant out-tuning years of refinement with a PyTorch stack built from the ground up. Across our 13 production tasks, this methodology did exactly that, with an average gain of 4.6%.

In This Article

1. Heuritech:
   The company behind this work
2. The Problem With Ad-Hoc Tuning:
   Why sequential one-at-a-time tuning produces biased conclusions
3. Scientific vs. Nuisance Parameters:
   The key insight that makes fair comparisons possible
4. Beyond Grid Search: Quasi-Random Sampling:
   Sobol sequences, and why not Optuna (yet)
5. Foundations Before Tuning:
   Metric, batch size, and compute budget
6. The Ordering Rationale:
   Why architecture → optimizer → scheduler → regularization
7. The Iterative Loop:
   Four phases of compounding gains on 13 production tasks
8. The 5-Phase Experiment Workflow:
   From research question to reproducible results
9. Results: From Matching TensorFlow to Beating It:
   +4.6% average, up to +14.5% on individual tasks
10. Making It Your Own:
    The distilled framework

Heuritech

A word about the system this work lives in:

Heuritech’s computer vision pipeline processes over 1.5 billion images sourced from 16 markets (US, Europe, Japan, and beyond) to help fashion & sportswear brands forecast demand more accurately and reduce overproduction.

The AI stack runs deep: object detection models identify garments in the wild, a dozen image classifiers label their attributes (shoe type, fabric, silhouette…), and from these visual signals we extract over 2,000 individual trends that get combined combinatorially (red × skirt × midi length) and mapped across geographies to produce more than 200,000 time series that we forecast into the future using advanced ensembling methods. We serve inference through Triton for production-grade throughput, use VLMs for specialized detections where traditional classifiers fall short, and run active learning pipelines to target our annotation budget (500K to 1M newly labeled images per month) where it matters most.

This article focuses specifically on one piece of that system: retraining our image classifiers, the dozen models that turn raw detections into the attribute labels everything downstream depends on.

For context on the rest of the pipeline, the team has written extensively across both sides of the stack:

On the vision side — directly upstream of this article — Paul Morel’s A social media image is worth a thousand fashion labels… walks through how we extract fashion attributes from social media images using deep learning, using metal-studs jackets as a worked example.

On the forecasting side — what happens downstream once these labels feed the time series — Etienne David’s three-part series:

1. Is fashion predictable? — the foundation: translating fashion dynamics into measurable time series.
2. Is Fashion Predictable? From Basic to Accurate Forecasting Models — applying statistical and deep learning forecasting models, with code.
3. Is Fashion Predictable? The Rise of Foundation Models — evaluating time series foundation models from Amazon, Microsoft, and IBM on fashion forecasting.

The Problem With Ad-Hoc Tuning

Most hyperparameter tuning in practice looks like this:

1. Pick a parameter (usually learning rate)
2. Try a few values
3. Pick the best one
4. Move to the next parameter
5. Repeat until deadline

This approach has four fatal flaws:

Confounded comparisons: Hyperparameters interact.
When you compare lr=1e-3 vs lr=1e-4 with weight decay (wd) fixed at wd=1e-5, you’re not comparing learning rates, you’re comparing:

• “lr=1e-3 with appropriate weight decay”
• “lr=1e-4 with the wrong weight decay.”

Each learning rate has its own optimal weight decay. A fair comparison requires tuning the parameters separately for each.

Incomplete exploration. Finding a good run isn’t enough, you need to find where performance degrades.
If your best result sits at the edge of your search range, you haven’t found the optimum; you’ve found the boundary of where you looked. Keep expanding until the best value is surrounded by worse ones on all sides.

Biased conclusions. If augmentation A gets 30 experiments and augmentation B gets 10, A may win simply because it was searched more thoroughly. Worse: if A’s sub-parameters are well-tuned but B’s aren’t, you’ll conclude A is better, when B might win with equal tuning budget. Fair comparison demands equal search effort across the options you’re evaluating.

Wasted compute. Grid search tests every combination, including those in unproductive regions. Random search is better but still inefficient.

Scientific vs. Nuisance Parameters: The Key Insight

The single most important concept in systematic tuning is the distinction between scientific and nuisance parameters.

Scientific parameters are what you’re trying to measure. They answer your research question directly.
Example: “What optimizer works best?” → The scientific parameter is optimizer and could take values sgd, adam, adamw and muon.

Nuisance parameters must be tuned per scientific parameter value to ensure a fair comparison. When comparing optimizer types, the learning rate is one of the nuisance parameters: different optimizers need different optimal learning rates. If you hold it constant, you might not be fairly evaluating the optimizer types.

Critically, nuisance parameters don’t have to be the same across scientific parameter values. When comparing SGD vs Adam, SGD’s nuisance parameter is learning rate (range 1e-3 to 1e-1), while Adam’s nuisance parameters are learning_rate (range 1e-5 to 1e-3) and beta1 (range 0.8 to 0.99). Each scientific value gets its own nuisance search space, tailored to what makes sense for that option. This is what makes the comparison fair.

The same applies to scheduler comparison: a constant learning rate has no nuisance parameters beyond LR itself, while cosine annealing has warmup epochs, restart multiplier, and decay length — all nuisance parameters that must be tuned to give cosine annealing a fair shot.

Fixed parameters are everything else, held constant to reduce the search space.

The Common Mistake

An easy mistake is to confuse nuisance parameters with scientific ones.
Here’s a real example. Suppose your research question is: “Which optimizer achieves the best validation metric: Muon or Adam?”

That question already tells you everything about the parameter classification, but in practice an easy mistake is to reach for the parameter suspected to have the most impact and place it as scientific:

In this setup, you’re treating a sub-parameter as the thing under study, and burying the actual decision — which optimizer? — as something to hold constant. The right setup flips this: the optimizer type is what you’re measuring, and each option gets its own nuisance search space.

Notice that the nuisance spaces are not just different values : they’re different parameters entirely. Muon doesn’t have a beta1, Adam doesn't have ns_steps.

Each optimizer gets tuned on its own terms:

• Muon:
  learning_rate ∈ [1e-3, 1e-1] ← search space is not shared
  ns_steps ∈ [3, 10]
  momentum ∈ [0.8, 0.95]
• Adam:
  learning_rate ∈ [1e-5, 1e-3] ← search space is not shared
  beta1 ∈ [0.8, 0.99]
  epsilon ∈ [1e-9, 1e-7]

Notice that even for nuisance parameters sharing the same name, their search spaces do not have to be shared across scientific parameters.

Now, the instinct to study learning rate isn’t wrong: it’s premature. Learning rate is one of the highest-impact continuous hyperparameters, and wanting to understand it deeply is exactly the right reflex. But the sequential logic matters: first, settle the optimizer choice (where LR is a nuisance parameter that must be tuned per optimizer to ensure a fair comparison). Then, once the optimizer is locked, that same instinct to study learning rate becomes the next scientific question : you’re now comparing LR schedules, warmup strategies, and decay shapes. The parameter is promoted from nuisance to scientific, in the right order.

This distinction seems pedantic until you realize it prevents the most common source of bias in ML experiments: unfairly comparing a well-tuned option against a poorly-tuned one.

Beyond Grid Search: Quasi-Random Sampling

Once you’ve defined your search space, how do you explore it efficiently?

To build intuition, consider a simple two-dimensional example: tuning two hyperparameters with a budget of just 16 experiments.

• One parameter has a high impact on our metric of interest (F1-score in this example): a narrow optimal region that’s easy to miss.
• The other has low impact : it matters, but not nearly as much.

Grid search places experiments on a rigid lattice. With 16 trials, you get a 4×4 grid, only 4 unique values per dimension. If the optimal region falls between two grid lines, every single experiment misses it. The heatmap tells the story: the grid’s best configuration scores 0.13, completely blind to the peak.

Random search breaks free from the lattice. Bergstra and Bengio (2012) showed that random search is more efficient than grid search precisely because it samples each dimension independently [2]. With 16 random trials, you get 16 unique values per dimension instead of 4. But randomness comes with a cost: clusters and gaps. Some regions get over-sampled while others are ignored entirely. Here, a lucky draw lands near the peak — best = 0.48, but coverage is uneven, and another random seed might miss it entirely.

Quasi-random sampling (Halton/Sobol sequences) combines the best of both worlds: the structural coverage of a grid with the dimensional independence of random search [3a]. Sobol sequences are designed to fill the space as uniformly as possible, no clusters, no gaps. Every new point is placed to maximally reduce the largest remaining hole in the coverage. The result: best = 0.51, with points distributed evenly across the entire space.

The marginal F1 plots on each figure make the mechanism visible. Look at the dots projected onto the high-impact parameter curve (top):

• Grid search gives you only 4 probes, none near the peak.
• Random search gives you 16, but they cluster on the left.
• Sobol gives you 16 evenly spaced probes, guaranteeing that at least one lands close to any narrow optimum.

Where It Really Matters: Higher Dimensions

The two-dimensional case is illustrative. The real payoff comes in higher dimensions.

In a 5-dimensional search with a grid of 5 values per dimension, you need 3,125 trials, but each dimension still only sees 5 unique values. That’s 3,125 experiments to test the same 5 learning rates. 80% of your compute is spent re-confirming what the low-impact dimensions do.

Random search with 30 trials gives you 30 unique values per dimension : 6× better coverage than the 3,125-point grid, at less than 1% of the cost.

Halton and Sobol sequences with 30 trials give you 30 unique values with guaranteed uniformity, no clusters, no gaps, reducing lucky-seed dependence. As dimensionality grows, the advantage of quasi-random over both grid and random search compounds dramatically.

Both SciPy’s Sobol and Halton samplers work the same way: generate uniform points in [0, 1], then scale to your parameter ranges. Here’s a minimal Sobol example with log-scaled learning rate and a categorical optimizer:

from scipy.stats.qmc import Sobol
import numpy as np

sampler = Sobol(d=2, scramble=True)
points = sampler.random(n=32)  # Sobol works best with powers of 2

lr = 10 ** (np.log10(1e-5) + points[:, 0] * (np.log10(1e-2) - np.log10(1e-5)))
optimizer = np.array(["sgd", "adam", "adamw"])[
    (points[:, 1] * 3).astype(int).clip(max=2)
]
# lr.shape = (32,)       → [2.1e-5, 7.4e-4, 3.1e-3, ...]
# optimizer.shape = (32,) → ["adam", "sgd", "adamw", ...]

And the Halton equivalent, same interface, but accepts any sample size and is naturally extensible (add trials by continuing the index):

from scipy.stats.qmc import Halton

sampler = Halton(d=2, scramble=True)
points = sampler.random(n=30)  # Halton: any n, not just powers of 2

lr = 10 ** (np.log10(1e-5) + points[:, 0] * (np.log10(1e-2) - np.log10(1e-5)))
optimizer = np.array(["sgd", "adam", "adamw"])[
    (points[:, 1] * 3).astype(int).clip(max=2)
]
# lr.shape = (30,)       → [1.2e-3, 4.8e-5, 6.0e-4, ...]
# optimizer.shape = (30,) → ["sgd", "adam", "adamw", ...]

Sobol vs. Halton :

Sobol sequences (used above) have slightly better theoretical uniformity in high dimensions (measured by discrepancy [3d], how evenly points fill a space), but require power-of-2 sample sizes for optimal coverage.

Visual example of Sobol sampling, from Wikipedia:

Halton sequences accept any sample size and are naturally extensible (you can add trials by continuing the index sequence, and new points automatically fill gaps in the existing coverage; however, regions added by widening the search space afterwards remain less explored).

Visual example of Halton sampling, from Wikipedia:

Both are vastly superior to grid or random search. If you anticipate extending a running experiment (widening a search range, adding more trials), Halton is the more practical choice.

Visual example of Random sampling, from Wikipedia:

How many trials?
As a rule of thumb: 5–30 trials per scientific parameter value, depending on the number of nuisance dimensions and the type of hyperparameter.

• With 1–2 nuisance parameters, 5–10 trials suffice.
• With 4–5, you need 20–30.
• Avoid tuning more than 5 hyperparameters simultaneously, above that, the space is too large for a reasonable trial budget to cover meaningfully (unless you are GPU rich!).

The type of parameter greatly impacts the number of trials needed. Continuous numeric parameters require the fewest trials (e.g. learning rate), then discrete numeric parameters (e.g. batch size, number of layers), and finally categorical parameters are the most costly — they will exceed the recommendations above as soon as they can take more than a couple of values.

In practice, we keep most experiments to 2–4 nuisance dimensions, and we try to exclude categorical parameters (or to treat them as scientific ones in later rounds of optimization).

Why Not Optuna or Hyperopt?

A natural question: if we’re trying to find the best configuration efficiently, why not use Bayesian optimization tools like Optuna[3b] or Hyperopt[3c]? They’re designed for exactly this : they model the objective function (our error metric) and focus trials on the most promising regions of each of our hyperparameters.

That’s precisely the problem. We’re not just trying to find the best configuration (yet) we’re trying to understand the system to direct our compute resources.

Bayesian optimizers tend to bias exploration toward the first good results they encounter. After a few trials, they build a surrogate model and start exploiting : concentrating new trials near the current best. This is efficient for finding a single optimum, but it leaves large regions of the search space unexplored. You can get a great configuration, but you learn almost nothing about why it’s great, or what else might work.

Quasi-random search doesn’t bias. Every region gets equal coverage, which means:

You can run all trials in parallel. Sobol generates every configuration upfront, so you can launch them all simultaneously across available GPUs. Bayesian optimization needs completed results to update its surrogate before sampling the next trial; you’re either waiting between runs or batching blindly, losing the adaptive advantage that justified using it in the first place.

You can distinguish high-impact from low-impact parameters. When all configurations near lr=1e-3 perform well regardless of weight decay, you know weight decay doesn't matter, and stop wasting time on it. Biased sampling makes this kind of sensitivity analysis unreliable.

You can extend a search range without starting over. If your best learning rate sits at the boundary, you need to explore further. With Sobol, you extend the range and add trials; all previous results remain valid data points. A Bayesian surrogate model built for the old range doesn’t transfer cleanly.

You can promote nuisance parameters to scientific ones. Sometimes you discover mid-experiment that a nuisance parameter actually matters. Because quasi-random search explored it uniformly, your existing data is already an unbiased sample of its effect. Bayesian optimization concentrated its exploration wherever the objective was high; you can’t draw fair conclusions about independent effects from biased coverage.

You build transferable knowledge. “Adam’s optimal LR for DINOv3[4a] backbones is broadly 1e-4 to 1e-3” is reusable across tasks and projects. “Optuna found lr=3.7e-4” is not.

Bayesian optimization has its place: once you understand your system, it’s powerful for squeezing out the last few points. But starting with it means optimizing a system you don’t understand, and that’s how you end up with fragile configurations that break when anything changes.

One notable exception, is when using a quasi-random sampler (like Sobol or Halton) specifically — for example, Optuna’s QMCSampler [4b]

Search Space Boundaries: A Critical Diagnostic

One diagnostic that’s easily missed in practice: always make your search space large enough to include clearly bad configurations at the edges. You want to see performance degrading at both boundaries: that’s how you know the optimum is enclosed, not just clipped by where you stopped looking.

If your best configuration is at the boundary of your search space, you haven’t found the best configuration — you’ve found the edge of where you looked.

This sounds obvious, but it’s an easy mistake to make : setting a learning rate range of [1e-4, 1e-3] and finding the best result at 1e-3. Is 1e-3 optimal, or would 2e-3 be better? You can’t know without expanding the range. Design search spaces to include the bad runs on purpose: they’re not wasted compute — they’re confirmation that you’ve explored far enough.

Sources:
[2] Bergstra & Bengio, 2012 (JMLR) : https://www.jmlr.org/papers/v13/bergstra12a.html
[3a] SciPy QMC Documentation : https://docs.scipy.org/doc/scipy/reference/stats.qmc.html
[3b] Optuna : https://github.com/optuna/optuna
[3c] HyperOpt : https://github.com/hyperopt/hyperopt
[3d] Discrepancy : https://en.wikipedia.org/wiki/Low-discrepancy_sequence
[4a] DINOv3 : https://arxiv.org/abs/2508.10104
[4b] Optuna QMCS https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.QMCSampler.html

Foundations Before Tuning

Before running a single experiment, three decisions shape everything that follows.

Know Your Compute Budget

Your compute budget shapes your experimental design. With 4 GPUs and 6-hour runs, you get ~20 experiments per week, enough for 2–3 nuisance parameters per round, not 5. This constraint is precisely why sequential tuning works: you can’t search a 10-parameter space on a realistic budget, but you can search three 3-parameter spaces in sequence.

Choose a Single-Number Metric

Whenever it’s possible to optimize for a single metric, do it.
A single target simplifies every decision in the tuning process: you always know whether a configuration is better or worse.

When you have multiple optimization targets, experiments become much harder to exploit. You’ll frequently find that improving one metric degrades another, making comparisons blurry and conclusions uncertain. Many experiments end up with ambiguous results that are hard to act on. A single number cuts through that ambiguity.

Your metric will depend on your task, but the principle is the same: collapse your evaluation to one number whenever the problem allows it. It’s not always possible, but when it is, it transforms the tuning process.

Set Batch Size First

Set batch size before anything else. Batch size is primarily a hardware efficiency decision, not a model quality decision.

Choose the largest batch size that maximizes GPU utilization.
The effect of batch size on optimization dynamics (larger batches → smoother gradients → potentially different optima) can almost always be compensated by adjusting the learning rate downstream. This means batch size doesn’t need to participate in the hyperparameter search — it’s a fixed precondition.

The notable exception: tasks where batch size has semantic meaning, such as contrastive learning, where the batch defines the ratio of negative samples. In those cases, batch size is a scientific parameter. For standard classification, it’s not.

The Ordering Rationale

The sequence in which you tune dimensions is not arbitrary : it’s chosen to minimize cascading retuning. The principle: tune the most globally impactful decisions first, and the most dependent decisions last.

Why does this work? Hutter et al. (ICML 2014) showed through functional ANOVA decomposition that even in high-dimensional HP spaces, most performance variance is explained by just a few parameters [4]. Bergstra and Bengio (2012) confirmed this with their effective dimensionality finding: for most tasks, only a small subset of hyperparameters truly matter, but which ones matter varies by task [2]. Wang et al. (2025) measured this directly on CNN image classification: architectural depth alone explained 38.5% of performance variance, learning rate 22.8%, dropout 13.1%, batch size just 1.5% [5]. Their Grouped Sequential Optimization Strategy, tuning groups in decreasing importance order, achieved a 32% reduction in total optimization time versus simultaneous search, with equivalent final accuracy.

The practical consequence is what we call cascading invalidation: changing a high-impact parameter shifts the optimal values of everything downstream, but not vice versa. Tuning in the right order means later decisions rarely invalidate earlier ones.

Architecture first. Architecture determines the entire function class, computational budget, and memory footprint. Changing the backbone invalidates optimizer settings, scheduler parameters, and regularization, everything downstream must be re-tuned. This is why Wang et al. found it explains the most variance: it’s the most global decision. Our approach: try the largest model that fits on your GPU for each architecture family you’re considering, then scale down if needed. This gives each family its best shot.

Optimizer second. The optimal learning rate for SGD (typically 1e-2 to 1e-1) differs from Adam’s (typically 1e-4 to 1e-3) by several orders of magnitude [1]. As the Google Tuning Playbook states: switching optimizers after tuning the learning rate wastes all prior LR work, the learning rate is effectively a different parameter for each optimizer. Fix the optimizer early, and everything you build on top stays valid.

Scheduler third. Scheduler parameters (warmup length, decay rate, restart multiplier) are tightly coupled to the optimizer and learning rate; they’re essentially a more granular tuning of the learning dynamics. Karpathy’s advice is emphatic: disable learning rate decay entirely until you’ve established the base learning rate, because the best schedule is sensitive to nearly every other optimization choice [6]. The Google Playbook recommends starting with a constant LR, then adding cosine or linear decay only once the base rate is locked [1].

Regularization last. Three reasons:

• First, regularization addresses the gap between training and validation metrics — you can only meaningfully measure that gap once architecture, optimizer, and scheduler are stable. Tuning regularization earlier means optimizing against a moving target.
• Second, large learning rates themselves act as implicit regularizers through optimization noise (Smith, 2018) [7]. The optimal dropout and weight decay values therefore depend on your learning rate: tune regularization before LR is set, and your results are misleading.
• Third, Hernandez-Garcia & König (2018) showed that data augmentation is robust to other HP changes, while weight decay and dropout are brittle, they can dramatically hurt performance if not carefully tuned in the context of the final configuration [8]. Regularization is the most dependent dimension, which is exactly why it comes last.

One interaction deserves special mention: learning rate × weight decay. Loshchilov & Hutter (2019) proved that L2 regularization and weight decay are not equivalent for Adam, which motivated AdamW [9]. More subtly, with BatchNorm layers, weight decay’s primary effect is modulating the effective learning rate rather than providing traditional regularization [10]. This means weight decay tuned before the learning rate is set will produce doubly misleading results, it’s partially an LR modifier masquerading as a regularizer.

Sources:
[1] Google Research Deep Learning Tuning Playbook ·
[2] Bergstra & Bengio, 2012 (JMLR) ·
[4] Hutter, Hoos & Leyton-Brown, 2014 (ICML) ·
[5] Wang et al., 2025, Grouped Sequential Optimization Strategy (arXiv:2503.05106) ·
[6] Karpathy, A Recipe for Training Neural Networks ·
[7] Smith, 2018, A Disciplined Approach to Neural Network Hyper-Parameters (arXiv:1803.09820) ·
[8] Hernandez-Garcia & König, 2018 (arXiv:1806.03852) ·
[9] Loshchilov & Hutter, 2019, Decoupled Weight Decay Regularization (ICLR) ·
[10] van Laarhoven, 2017, L2 Regularization versus Batch and Weight Normalization (arXiv:1706.05350)

The Iterative Loop: Building Systematically

You can’t tune everything at once, the search space is too large. A 5-dimensional search with continuous ranges requires orders of magnitude more trials than a 2-dimensional one. The solution is sequential: fix one dimension, tune the next, build on previous results.

The rest of this article applies that principle to a concrete case: Heuritech’s image classification pipeline. Our computer vision stack detects garments in social media images, then classifies their attributes through a set of hierarchical classifiers : one model identifies the type of garment (jacket, dress, top…), others label its texture, silhouette, pattern, color, and finer attributes like neckline shape or sleeve length. These predictions combine to define the thousands of fashion trends we track and forecast.

In total, we retrained 13 production classifiers spanning diverse visual tasks: outerwear type, dress type, shoe type, bag model, gender, brand logo, fabric, and more. Each task has different class distributions, visual complexity, and sensitivity to hyperparameters, which is precisely what makes them a good stress test for a general-purpose methodology.

One crucial design choice: We ran this entire progression on a single dataset, outerwear classification. The goal was never to find the best configuration for outerwear specifically. It was to find a general-purpose configuration that would work out of the box across all 13 production tasks, without any per-task tuning (our compute budget couldn’t afford tuning all 13 datasets directly). This is a much harder constraint, and it influenced decisions throughout. The other 12 datasets were held out entirely: the methodology was blind to them until final evaluation.

All results below are reported in p@r60 (precision at 60% recall): the precision our classifier achieves when recall is held at 60%. We need a recall floor for coverage, and we optimize precision above that floor : a single number, following the principle described earlier in this article.

As a reference point: DINOv3-ViTS16 with default PyTorch training parameters scored 0.650 p@r60 on our outerwear test set. Every phase below builds from there.

We followed a 4-phase progression (after the batch size and metric foundations described above), each phase building on the previous one’s best configuration:

Phase 1: Backbone Selection → p@r60 = 0.676

Scientific parameter: backbone architecture.
Nuisance: learning rate.
Fixed: constant scheduler, SGD optimizer, no regularization.

Result: DINOv3-ViTS16 selected as the foundation. Starting metric on our outerwear classification task: 0.676 p@r60, by far the best option among all the tested architectures!

Phase 2: Optimizer Choice → p@r60 = 0.692 (+1.6%)

Scientific parameter: optimizer (SGD, Adam, AdamW, Muon).
Nuisance: learning rate, betas, momentum, with different ranges per optimizer.
Fixed: DINOv3, constant scheduler, no regularization.

Result: Four optimizers, four different stories:

A quick note on experiment counts: the number of trials per optimizer is proportional to its nuisance dimensions, following the guidelines from the quasi-random sampling section. SGD has one nuisance parameter (learning rate), so it got fewer trials. Adam has three (learning rate, beta1, epsilon), so it required more trials to cover its larger search space. This isn’t unequal effort — it’s appropriately scaled effort. Each optimizer gets the budget its search space demands.

For SGD, we ran 4 experiments because we only had one nuisance parameter. SGD achieved the highest single-configuration score, but it had a very narrow optimal region, the peak was sharp and easy to miss:

Adam had more nuisance parameters to tune, so 15 experiments were needed. It scored lower but had a broad, flat optimal region of good learning rates:

Since we needed a single configuration that works across 13 classification tasks (shoe type, bag model, dress type, gender, etc.), Adam’s robustness mattered more than SGD’s peak performance. A configuration that’s 2% worse but works across all tasks beats one that’s 2% better on one task but fails on others.

Adam sensitivity analysis:

Recommended: Adam with lr=5e-4, beta1=0.86, epsilon=1e-3, beta2=0.999.

Phase 3: Scheduler Choice → p@r60 = 0.722 (+3.0%)

Scientific parameter: scheduler type (constant, linear decay, cosine annealing).
Nuisance: scheduler-specific parameters — constant LR has just the learning rate, while cosine annealing has warmup epochs, restart multiplier, decay length, and more.
With so many nuisance parameters, we increased the number of experiments to 30.
Fixed: DINOv3, Adam.

Result: Cosine annealing was the clear winner at 0.722 vs. constant LR at 0.699. The gain was not free, it required more epochs (training went from optimal at epoch 2–3 to optimal at epoch 10–20). But 3% relative improvement justified the additional compute (internally we mostly care about inference-time compute).

The first round of cosine experiments revealed an important diagnostic: the best configurations clustered at the boundary of our search space (especially for learning_rate), meaning we hadn’t explored far enough:

After extending the search space and doing our 30 experiments, we had a clearer picture, though the full scatter is dense. Filtering to the top 8 runs (>25% of our 30 experiments) makes the sweet-spot regions visible :

Recommended scheduler: Cosine Annealing with:

Phase 4: Regularization → p@r60 = 0.727 (+0.5%)

Scientific parameter: regularization strategy.
Nuisance: dropout, weight decay, image size.
Fixed: DINOv3, Adam, cosine annealing.

Result: Data augmentation did not improve results for our tasks — the sweet spot was simply dropout=0.10, image_size=224, weight_decay=0. The key takeaway: not every addition of complexity helps. Regularization should be validated, not assumed.

Regularization comes last for a reason: it addresses the gap between training and validation metrics. You can only meaningfully measure and reduce that gap once everything else — architecture, optimizer, scheduler — is stable. Tuning regularization earlier would mean optimizing against a moving target.

The Compound Effect

Each phase’s improvement was modest individually: +1.5%, +3%, +0.5%. But they compounded:

And that was just outerwear. On other tasks, the gains were larger: up to +14.5% on dress type classification.

The 5-Phase Experiment Workflow

The methodology above tells you what to tune. But in practice, even well-designed experiments fail: wrong YAML configs, unlogged parameters, GPU crashes wasting hours of compute. We developed a 5-phase workflow to make experiments reliable:

Phase 1: Collaborative Design

Start with a research question: “Does label smoothing improve DINOv3 accuracy?” Define the scientific parameter (label_smoothing, range 0.0–0.2), nuisance parameters (dropout, data augmentation), and fixed parameters (everything else). Research the literature for informed range choices.

Phase 2: Configuration Generation

A generation script creates all experiment configurations using quasi-random sampling. Critically, it also generates a TEST configuration: a single config with max_epochs=3, limit_train_batches=10, designed to validate the setup in minutes rather than hours.

Phase 3: Pre-Launch Verification

Before burning any real compute:

1. Verify hyperparameter logging: Run a verification script against the TEST config to confirm all scientific and nuisance parameters will appear in your experiment tracker. We use TensorBoard’s hparams tab: it logs every hyperparameter alongside training metrics, making post-experiment analysis possible. If a parameter isn’t logged, you can’t analyze it.

2. Verify code state: git status and git log to ensure you’re running committed code. Nothing is worse than discovering a great result came from uncommitted changes you can’t reproduce.

3. Run the TEST config: Launch the TEST yaml on a single GPU. This catches YAML syntax errors, missing dataset paths, shape mismatches, and configuration bugs. Catching these before launching 30 full experiments saves days.

Phase 4: Documentation

Write the experiment hypothesis before seeing results. This prevents post-hoc rationalization, the tendency to explain whatever result you got as if you expected it.

Our experiment documentation template includes:

• Objective (the research question)
• Hypothesis (what we expect and why)
• Configuration (dataset, model, metric, trial count)
• Parameter tables (scientific, nuisance, fixed — with ranges and rationale)

Phase 5: Launch and Monitor

After TEST validation passes, launch the full experiment. Distribute across available GPUs, monitor training curves for early divergence, and document results while they’re fresh.

The discipline of this workflow is what makes systematic tuning practical. Without it, the friction of running rigorous experiments pushes teams back toward ad-hoc tuning.

The Results: From Matching TensorFlow to Beating It

After applying this methodology across our full production workload, here are the results. Each task is a different image classification problem evaluated on precision-at-recall-60.

The TF Baseline column is not default TensorFlow, it’s the best TensorFlow configuration our team had refined over years of production use, based on an Xception backbone. This is the bar we needed to clear:

A few things to note:

Not every task improved : Brand logo and shoe type showed regressions of -2.9% and -1.8%. This is expected: the entire methodology was tuned on a single dataset and applied blindly to 12 others.
The fact that 11 out of 13 tasks improved (with an average gain of +4.6%) validates the general-purpose approach. But the two regressions point to the natural next step: now that we have well-characterized search spaces and a solid baseline configuration, we can run task-specific optimization on the underperforming tasks.
Data augmentation — which didn’t help on outerwear — may matter for brand logos, which have fundamentally different visual patterns. And this is precisely where Bayesian optimization earns its place: the exploration phase is done, the search spaces are mapped, and tools like Optuna can now squeeze out per-task gains within those spaces without risk of optimizing blindly.

Remember the starting point. On gender classification, our TensorFlow baseline (Xception backbone, refined over years) scored 82.5%. A naive PyTorch port with the same Xception architecture and default parameters scored 70.5% — a 12pp gap that made the migration look like a regression.

The final result — 91.0% — came from two compounding improvements: switching to a DINOv3 backbone (the architecture phase), and systematically tuning the training configuration (the methodology phases). Neither alone would have gotten there. The architecture gave us a better starting point; the methodology ensured we actually found its optimal operating point.

This is the broader lesson: an architecture upgrade without proper tuning leaves performance on the table, and tuning without the right architecture has a lower ceiling. The methodology’s value is making sure you reach whatever ceiling your architecture allows.

A telling sign of how unexpected these results were: when first presented, the team suspected a measurement error. The improvements, particularly the 10pp jump on gender classification, seemed too large to be real. This is exactly why the phased verification methodology matters: every result was traceable to a documented experiment with logged hyperparameters, committed code, and reproducible configurations. The skepticism was healthy; the evidence was airtight.

Making It Your Own

This methodology is framework-agnostic. The same principles apply whether you’re tuning PyTorch, TensorFlow, JAX, or anything else, and whether you’re recovering from a migration or starting from scratch.

Here’s the distilled framework:

The tools we built (Halton sequence generators, YAML configuration scripts, hyperparameter verification scripts, experiment documentation templates) are not complex, and the heavy lifting still runs on standard PyTorch. The value isn’t in the tools, it’s in the discipline of using them consistently.

Sources:
[1] Google Research Deep Learning Tuning Playbook :https://github.com/google-research/tuning_playbook
[1b] Goodfellow, Bengio & Courville, Deep Learning, Ch. 11.5: Debugging Strategies : https://www.deeplearningbook.org/

The Bottom Line

An experienced engineer’s intuition about learning rate might be right 80% of the time. But across 6 hyperparameters, 4 architectural choices, and 13 classification tasks, that 80% accuracy compounds to less than a coin flip. Systematic methodology beats intuition not because intuition is wrong, but because the investment in systematic methodology delivers compounding interest after each new experiment.

This approach gave us an average +4.6% over configurations our team had refined for years, not from a single insight, but from the compound effect of many small, rigorous, well-validated improvements. The two tasks that regressed (−2.9% on brand logos, −1.8% on shoe type) point to the natural next step: now that search spaces are mapped, Bayesian optimization can squeeze out per-task gains where the general-purpose config falls short.

Now it’s your turn to share: what’s your tuning methodology?

I’m particularly curious whether you’ve found techniques that transfer reliably across diverse tasks, the interaction between optimizer choice and task characteristics is something we’re still investigating.

Louis Develle is an ML Research Engineer at Heuritech, where he led the TensorFlow-to-PyTorch migration and designed the systematic tuning methodology described in this series.

Stop Guessing: A Systematic Methodology for Tuning Deep Learning Models was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

Welcome back everyone, I am Etienne David, a research engineer at Heuritech. Today, I would like to share a recent development in time series forecasting that has attracted significant attention in the research community: time series forecasting foundation models. In this article, we will start by explaining what a foundation model is. Then, we will review five foundation models recently released by major players in the field, such as Amazon, Microsoft, and IBM. Finally, as this is central to our work at Heuritech, we will evaluate how these new approaches perform on fashion forecasting tasks.

If you have not read them yet, I highly recommend taking a look at our two previous articles on fashion time series forecasting, they cover all the background needed for this article (Is Fashion Predictable?, Is Fashion Predictable ? From Basic to Accurate Forecasting Models). And as always, if you would like to reproduce the results and figures shown here, we have released all the code and materials in a public github repository.

1) Foundation models for time series forecasting.

A foundation model is a large machine learning model trained on massive amounts of data, designed to serve as a general-purpose model that can be adapted to a wide range of downstream tasks. The best-known examples come from text generation, with large language models such as ChatGPT from OpenAI, Claude from Anthropic, or Le Chat by Mistral. Trained on large-scale text data drawn from the internet, these models can now understand natural language and answer questions ranging from simple to complex with striking accuracy. It was only a matter of time before researchers began adapting foundation models to other domains, including time series forecasting.

Over the past few years, several companies have released foundation models specifically designed for generic time series forecasting. We list below some of them:

• Chronos 2: An encoder-only Transformer-based foundation model from Amazon with 120 million parameters, released on 17 October 2025. (Paper, Weights)
• TimesFM 2.5: Google’s decoder-only Transformer-based foundation model, released on 15 September 2025, trained on 200 million real and synthetic sequences. (Paper, Weights)
• Moirai 2: A lightweight decoder-only Transformer-based foundation model from Salesforce, released on 22 November 2025. It carries 11 million parameters and is pre-trained on 36 million real and synthetic time series. (Paper, Weights)
• FlowState: IBM’s state-space–based encoder-decoder foundation model, released on 19 August 2025, packing 9.1 million parameters. (Paper, Weights)
• TireX: An xLSTM-based foundation model from NXAI, released on 2 November 2025, with 35 million parameters trained over 47.5 million real and synthetic sequences. (Paper, Weights)

These foundation models are trained on vast sets of real and synthetic time series from diverse domains, such as retail, weather, and cloud infrastructure usage. The goal is to expose the models to as many examples as possible, enabling them to forecast new time series with high accuracy, regardless of forecasting horizon, seasonality, or noise. These approaches offer a major advantage: zero-shot forecasting. You no longer have to fit your own time series forecasting model. You can simply load a foundation model off the shelf and make predictions for any desired horizon. The only price to pay is size: some foundation models are huge, with hundreds of millions of parameters, and need a GPU to compute predictions efficiently.

But a key question remains: how accurate are these models? Let’s find out.

2) A Complete Code Base

Before testing the foundation models, we first introduce some code that makes it easy to experiment with the five models presented above. The complete code base is available at the following link. Using these models is straightforward. Apart from one or two exceptions, they all work in a similar way: a function to load the model weights from a Hugging Face repository, a predict function, and a save function for hosting the model’s weights. Several models also provide a fit function to fine-tune them and improve accuracy for specific use cases. We will come back to this later.

Below, we present the abstract class we defined to group the five foundation models. We also display the class implemented to wrap TimesFM 2.5, making it easy to use and experiment with. Note that in the init_model() method of the TimesfmForecastModel class, the model weights are loaded from the publicly released checkpoint available on Hugging Face.

import abc
import copy
import json
import os
from enum import Enum
from typing import Any, List

import numpy as np
import pandas as pd
from pydantic import BaseModel


from config import PREDICTION_ONE_YEAR, WEEK_FREQUENCY_TIMEINDEX
from timesfm import ForecastConfig, TimesFM_2p5_200M_torch

DEFAULT_MAX_CONTEXT = 256
DEFAULT_MAX_HORIZON = 256


class FoundationForecastModelType(str, Enum):
    chronos = "chronos"
    timesfm = "timesfm"
    moirai = "moirai"
    tirex = "tirex"
    flowstate = "flowstate"


class FoundationForecastModel(BaseModel, metaclass=abc.ABCMeta):
    """
        Abstract class for foundation models.
    """

    horizon: int = PREDICTION_ONE_YEAR
    model_type: FoundationForecastModelType = None
    model: Any = None
    model_args: List[str] = None

    class Config:
        arbitrary_types_allowed = True

    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        if self.model_args is None:
            self.model_args = []
        if self.model is None:
            self.init_model()

    @abc.abstractmethod
    def init_model(self):
        pass

    @abc.abstractmethod
    def save_model(self, path: str):
        pass

    def save_model_args(self, path: str):

        model_args = {arg_name: getattr(self, arg_name) for arg_name in self.model_args}
        model_args_path = os.path.join(path, "model_args.json")
        with open(model_args_path, "w") as f:
            json.dump(model_args, f, indent=4)

    def format_predictions(
        self, predictions: np.ndarray, last_historical_index: pd.Timestamp,
        prediction_columns: List[str]
    ) -> pd.DataFrame:
        predictions[predictions < 0] = 0.
        predictions = predictions.astype('float64')
        predictions_index = pd.date_range(
            start=last_historical_index, periods=self.horizon + 1, freq=WEEK_FREQUENCY_TIMEINDEX
        )[1:]
        return pd.DataFrame(
            predictions,
            columns=prediction_columns,
            index=predictions_index,
        )

    @classmethod
    def from_saved_model(cls, path: str, **kwargs) -> "FoundationForecastModel":

        model_args_path = os.path.join(path, "model_args.json")
        if os.path.isfile(model_args_path):
            with open(model_args_path) as f:
                model_args = json.load(f)
        else:
            model_args = {}

        if not os.path.isdir(path):
            raise ValueError(f"No model saved at following directory path : {path}")

        return cls(
            model_path=path,
            **model_args,
            **kwargs,
        )


class TimesfmForecastModel(FoundationForecastModel):
    model_type: FoundationForecastModelType = FoundationForecastModelType.timesfm
    model_path: str = "google/timesfm-2.5-200m-pytorch"
    model: TimesFM_2p5_200M_torch = None

    zero_shot: bool = None
    max_context: int = None
    max_horizon: int = None
    normalize_inputs: bool = None
    force_flip_invariance: bool = True
    infer_is_positive: bool = True

    class Config:
        arbitrary_types_allowed = True

    def init_model(self):
        self.zero_shot = True if self.zero_shot is None else self.zero_shot
        self.max_context = DEFAULT_MAX_CONTEXT if self.max_context is None else self.max_context
        self.max_horizon = DEFAULT_MAX_HORIZON if self.max_horizon is None else self.max_horizon
        self.normalize_inputs = True if self.normalize_inputs is None else self.normalize_inputs
        self.force_flip_invariance = True if self.force_flip_invariance is None else self.force_flip_invariance
        self.infer_is_positive = True if self.infer_is_positive is None else self.infer_is_positive

        self.model_args = [
            "zero_shot", "max_context", "max_horizon", "normalize_inputs", "force_flip_invariance",
            "infer_is_positive"
        ]

        self.model = TimesFM_2p5_200M_torch.from_pretrained(self.model_path, torch_compile=True)
        self.model.compile(
            ForecastConfig(
                max_context=self.max_context,
                max_horizon=self.max_horizon,
                normalize_inputs=self.normalize_inputs,
                force_flip_invariance=self.force_flip_invariance,
                infer_is_positive=self.infer_is_positive,
            )
        )

    def predict(self, historical_data: pd.DataFrame, time_index: str = None) -> pd.DataFrame:

        if time_index is not None:
            historical_data = copy.deepcopy(historical_data)
            historical_data.date_split(time_index)

        y_pred_df, _ = self.model.forecast(horizon=self.horizon, inputs=historical_data.values.T)
        predictions = y_pred_df.T
        return self.format_predictions(
            predictions, historical_data.index[-1], historical_data.columns
        )

    def save_model(self, path: str):

        self.model.save_pretrained(save_directory=path)
        self.save_model_args(path)

3) Foundation models and fashion time series forecasting

3.1) Zero-shot forecasting

Now that we have a functional code base, let’s assess the performance of the foundation models on a real-world fashion dataset introduced in our previous articles. This dataset contains 100 fashion time series, each representing the social media visibility of a sneaker type in the United States. We evaluate the five foundation models on a one-year forecasting task. The last year of each time series will be hidden, and zero-shot predictions will be generated using all five models.

Below is the portion of the code we released that was used to generate zero-shot foundation model predictions. Once again, it is remarkably straightforward to work with them:

import os

from model.foundation_model import (
    ChronosForecastModel, FlowStateForecastModel, FoundationForecastModelType, MoiraiForecastModel,
    TimesfmForecastModel, TirexForecastModel
)
from utils import (
    eval_predictions, load_forecast, load_time_series_csv, mean_errors_ranking, save_forecast
)

# Load the fashion time series data and remove the last year of data
fashion_ts = load_time_series_csv("data/us_female_sneakers.csv")
fashion_ts = fashion_ts.loc[:"2023-01-01"]
# Define foundations models
foundation_models = {
    FoundationForecastModelType.chronos.value: ChronosForecastModel(device_map='cpu'),
    FoundationForecastModelType.timesfm.value: TimesfmForecastModel(),
    FoundationForecastModelType.moirai.value: MoiraiForecastModel(),
    FoundationForecastModelType.tirex.value: TirexForecastModel(),
    FoundationForecastModelType.flowstate.value: FlowStateForecastModel(device='cpu')
}
# Generate zero-shot predictions.
models_forecasts = {
    model_name: model.predict(fashion_ts_train)
    for model_name, model in foundation_models.items()
}
# Save forecasts
save_forecast(models_forecasts=models_forecasts, forecasts_dir="model_forecasts")

Using the three error metrics introduced in our previous articles (MASE, MAPE, SMAPE), we evaluated the performance of the foundation models. Below, we show the average MAPE, SMAPE, and MASE across the 100 forecasts for each model:

TimesFM and Chronos achieve the best accuracy on the fashion forecasting use case, reaching the lowest values across all three metrics. TireX and FlowState are slightly less accurate but still show promising results. Only Moirai appears to underperform on this specific dataset. To put these results into perspective, the table below compares the foundation models with some of the best deep learning methods we tested in our previous article.

One of the most striking findings is that foundation models like Chronos and TimesFM, even in a zero-shot setting, achieve performance comparable to deep learning models that were specifically trained on our fashion dataset. This demonstrates the remarkable ability of these models to generalise to new time series without any task-specific training.

The forecast examples below illustrate how these models capture both complex seasonal patterns and the overall trends of the fashion items.

3.2) Fine-tuning a foundation model

In section 2, we mentioned that foundation models can also be fine-tuned to enhance their accuracy on a specific use case. Thus, to assess the maximum performance of the Chronos model, we fine-tuned it on our dataset of sneaker time series. We show its accuracy results below.

The results of Chronos after being fine-tuned on our dataset are striking. The model achieves the best accuracy across all three error metrics and outperforms the top deep learning models we tested in a previous article. Below, we present two graphs comparing predictions of the two Chronos models, highlighting how the fine-tuned version better captures the specific patterns and nuances of our sneaker time series.

4) Conclusion

In this chapter, we continued our exploration of fashion time series forecasting by evaluating a new class of models called foundation models. These approaches are highly promising:

• They are simple to use, with pre-trained weights publicly available on Hugging Face.
• They can be used for time series zero-shot forecasting.
• They can be fine-tuned to enhance their accuracy on specific use cases.
• On our fashion forecasting task, Chronos and TimesFM yield highly competitive results.

While these approaches are still relatively new, their potential is clear. We can be optimistic about future improvements, and we will continue to closely follow advances in this research field.

To conclude this chapter, we would like to highlight a recent paper by the Amazon Research team introducing a new benchmark dataset, called Fev-Bench, designed to evaluate and rank time series forecasting models, including foundation models. We were pleasantly surprised to see that we were cited in this paper, as a sample of Heuritech Fashion time series was included in Fev-Bench, giving our data a role in this reference set. This means that future models may encounter our fashion forecasting use case during evaluation. Let’s see how the next generation of foundation models rises to the challenge!

Hope you enjoyed this third post! If you would like to reproduce the graphs and results presented in this article, you can access the linked code repository on github. For more content like this, don’t forget to follow me and Heuritech.

Is Fashion Predictable? The Rise of Foundation Models was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

Hi everyone, I’m Etienne DAVID, a research engineer at Heuritech. Today, we’ll continue exploring fashion forecasting through the lens of mathematical models and AI.

If you haven’t read my first article on this topic yet, you can find it here. In that article, I discuss how we analyze the evolution of thousands of fashion items over time by leveraging social media data and visual recognition algorithms. For a deeper understanding of the visual recognition aspect, I recommend checking out this insightful article by my colleague, Paul Morel.

Today, I’d like to take things a step further. First, we’ll review some of the most powerful statistical and deep learning models for forecasting. Then, I’ll share complete code examples to demonstrate how to apply these models to any type of signal. Finally, we’ll test their performance on a dataset of fashion time series, tracking the evolution of 100 sneakers in the United States.

As always, if you’d like to replicate the graphs and results featured in this article, you can access the linked code repository here.

1) From Basic to Accurate Forecasting Models

In the previous article, we presented two very simple methods: the snaive method and the exponential smoothing method. In this section, we’ll review several of the most well-known and effective forecasting models from the literature. These models can be classified into two main categories:

1. Statistical Models: These mathematical methods model the distribution of time series data. They typically require minimal tuning, offer explainable predictions, and provide statistical guarantees.
2. Deep Learning Models: The latest state-of-the-art forecasting models fall into this category. With their ability to handle large datasets and integrate diverse, heterogeneous multivariate data, they have achieved unprecedented accuracy across a wide range of applications and datasets.

Let’s start with the statistical models.

SNAIVE and ETS:
We already introduced these two models in our previous article. Both are simple approaches that can be quickly implemented. A good practice is to start any forecasting problem with them, as they serve as benchmarks for evaluating the performance of more complex models in future experiments.

PROPHET (paper, overview):
Developed by Meta in 2017, Prophet is one of the most popular tools for time series forecasting and is widely used across industries. Its intuitive design allows users to produce reliable forecasts with minimal effort, even when dealing with complex seasonality and missing data.

THETAM (paper):
The THETA model is a simple yet effective forecasting approach that decomposes a time series into different “theta lines” by applying theta coefficients. This highlights various aspects of the data, such as trends and seasonality. Typically, two theta lines are used: one capturing the long-term trend (often modeled with linear regression) and another focusing on short-term fluctuations (handled via simple exponential smoothing). The forecasts from these components are then combined to produce the final prediction.

TBATS (paper, overview):
The TBATS model — short for Trigonometric, Box-Cox transformation, ARMA errors, Trend, and Seasonal components — is a complex but highly accurate forecasting method. Thanks to its trigonometric multi-seasonality, it effectively handles datasets with multiple seasonal patterns, making it suitable for complex time series data.

ARIMA (paper, overview):
ARIMA (AutoRegressive Integrated Moving Average) is arguably the most well-known statistical forecasting method. Numerous variations — such as ARMA, SARIMA, and ARIMAX — exist to address a wide range of forecasting tasks. While selecting the appropriate variation and parameters can be challenging, many modern packages offer automated ARIMA implementations that identify the best model configuration for your time series.

While the previous statistical approaches have many advantages — they are simple to use, require minimal data for calibration, and provide interpretable predictions with a measure of confidence— they can become computationally expensive when applied to large datasets. As a result, over the past decade, a number of deep learning models have been introduced, delivering impressive results across a variety of datasets. Here, we’ll present some of them.

DeepAR(paper):
The DeepAR model, introduced in 2019 by researchers at Amazon, uses neural networks to parametrise probabilistic distributions. These distributions are then used to simulate predictions during inference. Based on Recurrent Neural Networks (RNNs), DeepAR was one of the first deep learning-based models to achieve high accuracy with probabilistic forecasts across a broad range of real-world datasets.

N-BEATS/N-HiTS (paper/paper):
N-BEATS and N-HiTS are powerful deep learning models that rely on blocks of Fully Connected Networks (FCNs). These models are grouped together because N-HiTS, released in 2022, is an enhanced version of N-BEATS, which was introduced in 2020. Today, N-HiTS is regarded as one of the most advanced forecasting methods, particularly well-suited for large datasets.

PatchTST (paper):
In my opinion, the PatchTST model is one of the best forecasting models currently available in the literature. It uses a Transformer-based architecture and has delivered impressive accuracy across several benchmark datasets. Released in 2023 by IBM, PatchTST is the first approach to successfully adapt Transformer networks — typically used for Natural Language Processing (NLP) — to time series forecasting.

TSMixer/TimeMixer (paper/paper):
TSMixer and TimeMixer are two of the most recent forecasting models introduced in the literature. Both models share a similar concept: pre-training a deep learning model on a large collection of time series from diverse and heterogeneous sources. These models aim to create a robust system capable of forecasting a wide range of signals by learning patterns across multiple domains. Once pre-trained, they can be applied to specific forecasting tasks or further fine-tuned on targeted datasets to improve performance.

2) A Complete Code Base

Now that we’ve introduced several forecasting models, this section will present a comprehensive codebase that integrates all of these approaches into a unified framework available here.

We begin by introducing the abstract class ForecastModel, which defines a general structure for all the forecasting models.

from abc import ABC, abstractmethod
from typing import Optional, Union

import pandas as pd
from config import PREDICTION_ONE_YEAR
from pydantic import BaseModel


class ForecastModel(BaseModel, ABC):
    """
    Abstract class for a forecasting model.
    """

    horizon: int = PREDICTION_ONE_YEAR
    model_type: Optional[str]

    @classmethod
    @abstractmethod
    def from_saved_model(cls, path: str, **kwargs):
        pass

    @classmethod
    @abstractmethod
    def save_model(cls, path: str):
        pass

    @abstractmethod
    def predict(self, historical_data: pd.DataFrame, time_index: str = None) -> pd.DataFrame:
        """Return the prediction of a forecasting model.

            Args:
                historical_data: A pd.DataFrame gathering time series signals with date as index and time series' name as columns.
                time_index: A date in the format 'YYYY-MM-DD' where to stop the historical signals and start the
                 forecasts. If None is provided, the forecast period will start at the end of the time
                 series.

            Returns:
                A pd.DataFrame gathering the predictions.

        """

    @abstractmethod
    def fit(self, historical_data: pd.DataFrame, time_index: Optional[str] = None):
        """
            Model fitting function
        """
        pass

Using the Statsforecast package developed by Nixtla, along with a second abstract class, StatsForecastModel, we’ve implemented almost all the statistical models discussed earlier as follows:

import statsforecast.models as statsforecast_models


class EtsForecastModel(StatsForecastModel):
    """
        Forecast model using AutoTbats method https://nixtlaverse.nixtla.io/statsforecast/src/core/models.html#autotbats
    """
    model_type: StatsForecastModelType = StatsForecastModelType.ets
    model_config: str = "ZZZ"
    damped: Optional[bool] = None
    phi: Optional[float] = None

    def init_model(self):
        self.model = statsforecast_models.AutoETS(
            season_length=self.season_length,
            model=self.model_config,
            damped=self.damped,
            phi=self.phi
        )


class TbatsForecastModel(StatsForecastModel):
    """
        Forecast model using AutoTbats method https://nixtlaverse.nixtla.io/statsforecast/src/core/models.html#autotbats
    """
    model_type = StatsForecastModelType.tbats
    use_boxcox: Optional[bool] = None
    use_trend: Optional[bool] = None
    use_damped_trend: Optional[bool] = None
    use_arma_errors: Optional[bool] = None

    def init_model(self):
        self.model = statsforecast_models.AutoTBATS(
            season_length=self.season_length,
            use_boxcox=self.use_boxcox,
            use_trend=self.use_trend,
            use_damped_trend=self.use_damped_trend,
            use_arma_errors=self.use_arma_errors
        )

Following the same approach, we implemented all the deep learning models previously mentioned using the Neuralforecast package by Nixtla, along with an abstract class, NeuralForecastModel:

import neuralforecast.models as neuralforecast_models


class NHitsForecastModel(NeuralForecastModel):
    """
        Forecast model using NHits method https://nixtla.github.io/neuralforecast/models.nhits.html

        Args:
            n_blocks: List[int], Number of blocks for each stack. Note that len(n_blocks) = len(stack_types).
            mlp_units: List[List[int]], Structure of hidden layers for each stack type. Each internal list should contain the number of units of each hidden layer. Note that len(n_hidden) = len(stack_types).
            n_freq_downsample: List[int], list with the stack’s coefficients (inverse expressivity ratios). Note that len(stack_types)=len(n_freq_downsample)=len(n_pool_kernel_size).
            n_pool_kernel_size: List[int], list with the size of the windows to take a max/avg over. Note that len(stack_types)=len(n_freq_downsample)=len(n_pool_kernel_size).
    """

    model_type = NeuralForecastModelType.nhits

    n_blocks: List[int] = [1, 1, 1]
    mlp_units: List[List[int]] = 3 * [[512, 512]]
    n_pool_kernel_size: List[int] = [2, 2, 1]
    n_freq_downsample: List[int] = [4, 2, 1]

    def init_model(self):

        model = neuralforecast_models.NHITS(
            input_size=2 * self.horizon,
            h=self.horizon,
            max_steps=self.max_steps,
            learning_rate=self.learning_rate,
            batch_size=self.batch_size,
            random_seed=self.seed,
            n_blocks=self.n_blocks,
            mlp_units=self.mlp_units,
            n_pool_kernel_size=self.n_pool_kernel_size,
            n_freq_downsample=self.n_freq_downsample,
            stack_types=["identity"] * len(self.n_blocks),
            num_workers_loader=self.num_workers_loader
        )
        self.model = NeuralForecast(models=[model], freq=WEEK_FREQUENCY_INDICATOR)


class PatchTSTForecastModel(NeuralForecastModel):
    """
        Forecast model using PatchTST method https://nixtla.github.io/neuralforecast/models.patchtst.html

        Args:
            encoder_layers: int, number of layers for encoder.
            n_heads: int, number of multi-head’s attention.
            hidden_size: int, units of embeddings and encoders.
            linear_hidden_size: int, units of linear layer.
    """

    model_type = NeuralForecastModelType.patchtst

    encoder_layers: int = 3
    n_heads: int = 16
    hidden_size: int = 128
    linear_hidden_size: int = 256
    patch_len: int = 16

    def init_model(self):

        model = neuralforecast_models.PatchTST(
            input_size=2 * self.horizon,
            h=self.horizon,
            max_steps=self.max_steps,
            learning_rate=self.learning_rate,
            batch_size=self.batch_size,
            random_seed=self.seed,
            encoder_layers=self.encoder_layers,
            n_heads=self.n_heads,
            hidden_size=self.hidden_size,
            linear_hidden_size=self.linear_hidden_size,
            patch_len=self.patch_len,
            num_workers_loader=self.num_workers_loader
        )
        self.model = NeuralForecast(models=[model], freq=WEEK_FREQUENCY_INDICATOR)

The Statsforecast and Neuralforecast packages are intuitive and powerful tools, consistently maintained and updated by the Nixtla team with the latest forecasting models from the literature. I strongly recommend using them for your forecasting tasks. While I haven’t gone into extensive detail on the code, if you’re interested in exploring it further, I encourage you to check out the complete codebase here and experiment on your own. Now, let’s shift back to our fashion forecasting problem and evaluate how accurately these methods can predict fashion trends.

3) Fashion time series forecasting

To conclude this second article, we’ll test the models introduced earlier on a dataset of fashion time series, which tracks the evolution of 100 sneaker types in the United States. These time series represent the weekly visibility on social media of 100 sneakers in the U.S., from October 10, 2015, to January 1, 2024. For the forecasting task, we’ll focus on predicting one year ahead. Specifically, we will hide the data for 2023 and, using the remaining historical data, evaluate how well the forecasting models predict the 52 weeks of 2023. Below is an example using the Matte Leather Sneakers. The black portion of the time series represents the data used to train the forecasting models, while the grey portion represents the ground truth (test set), which is hidden during training and used to calculate prediction errors.

Using the previously presented codebase, here’s the script we used to train our forecasting models, generate predictions for the 100 fashion time series, and evaluate the accuracy of these predictions.

from neuralforecast_model import (
    DeepARForecastModel, NBeatsForecastModel, NeuralForecastModelType, NHitsForecastModel,
    PatchTSTForecastModel, TimeMixerForecastModel, TSMixerForecastModel
)
from prophet_model import ProphetForecastModel, ProphetForecastModelType
from snaive_model import SnaiveForecastModel, SnaiveForecastModelType
from statsforecast_model import (
    ArimaForecastModel, EtsForecastModel, StatsForecastModelType, TbatsForecastModel,
    ThetaForecastModel
)
from utils import (
    eval_predictions, load_time_series_csv, mean_errors_ranking
)

# Load the fashion time series data
fashion_ts = load_time_series_csv("us_female_sneakers.csv")
fashion_ts_train = fashion_ts.loc[:"2023-01-01"] # Split the dataset into training data up to January 1, 2023

# Define statistical models with their configurations
statistical_models = {
    SnaiveForecastModelType.snaive:
        SnaiveForecastModel(),
    ProphetForecastModelType.prophet:
        ProphetForecastModel(),
    StatsForecastModelType.ets:
        EtsForecastModel(model_config="AAA", damped=True),
    StatsForecastModelType.theta:
        ThetaForecastModel(),
    StatsForecastModelType.tbats:
        TbatsForecastModel(
            use_boxcox=True, use_arma_errors=True, use_damped_trend=True, use_trend=True
        ),
    StatsForecastModelType.arima:
        ArimaForecastModel(
            d=1,
            D=1,
            start_p=1,
            max_p=1,
            start_q=1,
            max_q=1,
            start_P=0,
            max_P=0,
            start_Q=2,
            max_Q=2
        )
}

# Define deep learning-based forecasting models
deep_models = {
    NeuralForecastModelType.deepar: DeepARForecastModel(),
    NeuralForecastModelType.nbeats: NBeatsForecastModel(),
    NeuralForecastModelType.nhits: NHitsForecastModel(),
    NeuralForecastModelType.patchtst: PatchTSTForecastModel(),
    NeuralForecastModelType.tsmixer: TSMixerForecastModel(),
    NeuralForecastModelType.timemixer: TimeMixerForecastModel(),
}

# Fit each deep learning model using the training data
for model in deep_models.values():
    model.fit(fashion_ts_train)

# Generate forecasts from both statistical and deep learning models
models_forecasts = {
    **{
        model_name: model.predict(fashion_ts_train) for model_name, model in statistical_models.items()
    },
    **{
        model_name: model.predict(fashion_ts_train) for model_name, model in deep_models.items()
    }
}

# Evaluate the forecasts using the metrics MASE, SMAPE and MAPE
error_metrics = eval_predictions(fashion_ts, models_forecasts)

# Rank the models based on mean error metrics and save the results to a CSV file
mean_errors_ranking(error_metrics=error_metrics, output_path="mean_errors_ranking.csv")

3.1) Computational time

Before assessing the accuracy of the different methods, let’s first take a quick look at the computational time for each approach. The table below shows the training and inference times for each forecasting model.

Two observations can be made. First, since statistical models require training for each new time series, the training time is equivalent to the inference time. This makes methods in this category slower when applied to large datasets. For example, the ARIMA method takes 30 minutes to forecast our dataset of 100 time series, which makes using such a model on a dataset with thousands of time series costly.

Second, a single deep learning model can be trained on an entire dataset. In this article, we’re working with a small sample of time series, which is why the training time for deep learning models is relatively short. However, with larger datasets, training these models can take considerably longer. The advantage, though, is that once trained, models can be saved and reused for inference, making them highly efficient at computing prediction on new time series. That said, deep learning models require careful training, including optimising hyperparameters and regular retraining to ensure they continue to perform at their best.

3.2) Average Error Ranking

Using the three error metrics introduced in the previous article (MASE, MAPE, SMAPE), we evaluated the performance of the various forecasting methods. Below, we present the average MAPE, SMAPE, and MASE across the 100 forecasts for each model:

We can see that the deep learning model PatchTST achieves the best average accuracy across the three metrics. Two statistical models, TBATS and Prophet, also demonstrate promising results, ranking 4th and 5th when considering the MASE metric. However, two deep learning models, DeepAR and TimeMixer, perform poorly. This doesn’t suggest these methods are unsuitable for our forecasting task, but rather that they were not properly calibrated. We used the same training parameters for all deep learning models, but ideally, we should have conducted a full grid search for each model individually. This underscores the fact that deep learning models are powerful tools, but they require careful tuning.

In addition to the average results, we can also evaluate the distribution of errors for each method. Below is a graph displaying the MASE distribution for the tested forecasting models. As we can see, PatchTST consistently provides better predictions, with its MASE errors generally lower than those of the other methods.

Here are some forecast examples. We can observe that many methods effectively capture the various seasonal patterns and the overall trends of the fashion items.

4) Is fashion predictable?

Looking at the outliers, we can see that a few time series were not accurately predicted by any of the tested methods. One such example is the White Retro Football Sneakers. Below, we present the time series along with the predictions from each model.

We can observe that the visibility of this sneaker skyrockets in 2023, yet all the forecasting models fail to capture this sudden increase. Upon reviewing the historical data, there are no early signals or patterns that could have helped even the most advanced mathematical methods predict such a change. Without incorporating external signals, like fashion shows or influencer behaviour, these fashion evolutions remain beyond the reach of mathematical models. This will be the focus of the next article.

Hope you enjoyed this second post! If you’d like to reproduce the graphs and results presented in this article, you can access the linked code repository here. For more content like this, don’t forget to follow me and Heuritech. Your support means a lot, and we’d love to have you back for future posts!

Is fashion predictable ? part 2 was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

In a recent blogpost, Étienne David introduced how it is possible to forecast a fashion trend based on the time series representing its visibility on social media. In case you missed it, you can read it here.

I’m Paul Morel, Data Science Lead at Heuritech. In this article, I will present how we extract meaningful insights from social media images using Deep Learning. Together, we will illustrate the detection of a fashion trend using the example of metal studs jackets for men.

Computer Vision at Heuritech

Each fashion image on social media contain many layers of valuable insights. At Heuritech, our taxonomy enumerates more than 2,000 clothing attributes (including colors, shapes, types and other attributes) detected on more than 750 million images. By crossing these attributes, we designed a catalog of more than 85,000 fashion trends.

For our example trend, we need 3 distinct Deep Learning models:

• One detecting the fashion items, including the human and the jacket.
• A second detecting the clothing attributes of the jacket, including the metal studs.
• A third inferring the gender of the human.

To be able to do that, several considerations are key to understanding how we handle fashion attributes detection, which makes Heuritech unique.

Exploring fashion with AI and social media images

Defining and Predicting Specific Classes

Let’s focus on how we define and learn to predict a specific class for our clothing attributes detector, such as metal studs. This model performs image classification, identifying one or more classes among those it’s been trained to recognize. Most of Heuritech models are doing classification.

Acquire the right data, for the right classes

The first step is defining classes that make sense for a fashion expert. Keep in mind that these classes will be combined with other ones to define our fashion trends. These classes must be:

• Granular enough to define and distinguish trends properly, avoiding confusion.
• Exhaustive to cover all types and attributes of a given clothing while ensuring classes are mutually exclusive, i.e. there is no overlap in the classes definition.
• Visible enough to get decent volumes of high-quality samples for training.

One key challenge is avoiding false detections, such as confusing metal studs with rhinestones or pearls. Below are examples of attributes that must not be confused with metal studs.

One key challenge is avoiding false detections, such as confusing metal studs with rhinestones or pearls. Below are examples of attributes that must not be confused with metal studs.

We tackle this issue in two ways:

1. Learning these classes if they’re useful for detecting other trends.
2. Training the model with negative samples of metal studs. However, defining all possible classes a model might “see” is often impractical (too expensive in tagging as there would be too many classes), leading us to rely on open set learning. This field allows models to predict confidently even when encountering unforeseen classes.

Data quality is king and defining accurately the classes as well as finding the suitable training samples is a long, meticulous job performed by our Fashion experts. Shoutout to them!

Building a hierarchy of classes

Our classification models don’t simply only pick a single class from the list of trained classes. Many of our models involve both:

• Classifiers parts detecting single features (e.g., type of top).
• Multilabel identifying multiple attributes (e.g., sleeve length, neckline shape).

At the same time. Classes may also have hierarchical relationships. For instance, our clothing attributes model identifies specific denim types (e.g., bleach, tie-dye) while consolidating them under a broader “denim” class. If confidence is low, the model predicts only the broader class.

For metal studs, no complex hierarchy is required. However, if we wanted a broader trend like “jacket embellishments for men”, we could consolidate predictions of metal studs, pearls, rhinestones, etc., into a parent class.

Adjust the training process to fit our need

Now that we have the right data for the right model architecture, we train and evaluate the model. A classification model usually predicts a score for every single class between 0 and 1, representing its confidence in detecting the given class. During training, scores above 0.5 are considered detections, and the model is corrected if it makes errors. This threshold is later fine-tuned using a validation set to maximize performance metrics.

The threshold value impacts 2 important metrics, precision and recall:

• A higher threshold increases precision but lowers recall.
• A lower threshold improves recall but risks more false positives, harming precision.

Chosing this threshold value is a tradeoff that needs to be made for each class. The predictions of our models are aggregated to build trend lines, and we want them to be insightful while having enough volume so they can be statistically representative of the given trend.

At Heuritech, we value precision twice as much as recall. To balance this, we use the F-β score, an adjusted harmonic mean of precision and recall where β reflects the weight of recall. For instance, using β = 0.5 prioritizes precision twice as much as recall.

This score is computed for a wide range of possible threshold values, so we can use it to select the threshold that maximizes it.

Below curve illustrates how precision, recall, F-β and detected volume of metallic studs on clothes evolve with threshold changes, across our valdation set. The optimal threshold maximizes the F-β score while maintaining adequate detected volume.

Vertical plain and dashed lines represent 2 distinct values of threshold, the one from the model during a previous training (dashed line) and the latest one (plain line). By looking at the intersection with the colored lines, we can find the associated values of the various metrics measured (precision, recall, etc.). Doing so, we see that it is useful to slightly increase the former threshold from ~0.7 to 0.75 to maximize the F-β score.

We can notice here that the new threshold is the one where the F-β metric is the highest, although we could have better precision by increasing the threshold value. Yet, this would harm too much the predicted volume of detected metallic studs so it is preferrable to keep a non extreme threshold value.

Provided that we are able to ensure the model has good performance (which is no simple thing, as usual with Deep Learning), we should be good to go to start detection of metal studs.

Building a pipeline of model to track fashion trends

While we’ve discussed training a single class within a classification model, fashion insights require multiple models working together. Unlike trendy large foundation models, we use a modular pipeline:

• Image-wise models: Detect fashion objects (e.g., humans, tops, dresses) and classify context (e.g., outdoor vs. indoor, gym).
• Item-specific models: Analyze detected items for features like attributes, colors, and patterns. For instance, a top classifier detects sleeve length, neckline shape, and type. We also perform regression to spot the color of the clothes.
• Aggregation: Combine predictions to define trends, such as red V-neck shirts for women.

By aggregating the images and predictions for a specific geography, we can track a given fashion trends worldwide. Our modular design allows independent updates to specific scopes, such as adding new patterns without retraining unrelated models. Below image showcases the behavior of our deep learning pipeline.

Conclusion

Building deep learning models for fashion involves multiple steps and expertise. This article touched on:

• Training classification models.
• Avoiding common pitfalls like false detections.
• Using modular pipelines for modularity.

For the sake of simplicity, we didn’t cover many problems that can show up in the detection of trends, even the most obvious ones: how do we ensure the stability and improvement of a given model from one training to an other? How to control the impact of a model on the related trends? Etc.

If you are technical or simply curious about it, please let me know below if you are interested about a detailed article related to this below, by giving it a 👍 or letting me know in the comments below!

Also, we didn’t tackle the topic of analyzing hundreds of millions of images in reasonable amounts of time and costs, which is a major topic for our solution to work properly. Indeed, modularity is one thing, but our approach needs also to be scalable. This will be the topic of an upcoming article about Computer Vision Model Serving using NVIDIA AI Triton, stay tuned!

Last but not least, if you’re passionate about data science and looking for an exciting opportunity, we’re hiring! You can take a look at the job description here.

A social media image is worth a thousand fashion labels… was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

I know that with a title like that, I may receive some negative (or positive?) comments if designers or luxury brands come across my article. Anyway, I’m Etienne DAVID, research engineer at Heuritech and today, we will explore the complex subject of fashion forecasting through the lens of mathematics, machine learning and AI.

Fashion is a complex and ever-changing sector, especially in recent decades with the emergence of “Fast Fashion” and the consequent acceleration of trend cycles. To understand and predict fashion dynamics using mathematical approaches, the cornerstone will be the data: How can we translate the evolution of fashion into measurable values?

1) Translate fashion to time series

In response to this first question, Heuritech, a cutting-edge fashion technology company based in Paris, has proposed a surprising solution: analyse images on social media to illustrate the evolution of fashion. The following approach is used:

Here is an example of a Heuritech time series representing the appearance of Chunky Sneakers on social media in the United States from January 1, 2017, to January 1, 2023:

As social media data has several biases, we normalise the raw time series above to more accurately represent the fashion evolution of Chunky Sneakers. For instance, if you look at the raw time series at the beginning of 2020, you’ll notice a sudden decrease that may be due to COVID “stay-at-home” quarantines (resulting in fewer outdoor pictures on social media). This effect has been completely removed from the normalised signal shown below:

2) Fashion time series forecasting

Now that we successfully translated the evolution of fashion into time series, building mathematical predictions is straightforward. Let’s start with two simple statistical models: the seasonal naïve method and the exponential smoothing method (Holt-Winters’ additive version). See this great online book for a complete description of these models.

We provide below Python classes implementing the two models:

import pandas as pd
from statsmodels.tsa.api import ExponentialSmoothing
from pydantic import BaseModel

WEEKS_IN_YEAR = 52
HORIZON = 52


class EtsForecastModel(BaseModel):
    """Class defining an ETS (exponential smoothing) model.

        Args:
            season: define the seasonality length.
            horizon: define the forecast horizon of the Ets model.
    """

    season: int = WEEKS_IN_YEAR
    horizon: int = HORIZON

    def fit_predict_single_ts(self, ts_signal: pd.Series) -> pd.Series:
        """ Fit ETS models and return its prediction.

            Args:
                ts_signal: A pd.Series gathering a single time series signal.

            Returns:
                single_forecast: A pd.Series gathering the ETS prediction.
        """

        model = ExponentialSmoothing(
            ts_signal, seasonal_periods=self.season, seasonal="add", trend="add"
        )
        fitted_model = model.fit()
        single_forecast = fitted_model.forecast(self.horizon)
        single_forecast.name = ts_signal.name
        return single_forecast


class SnaiveForecastModel(BaseModel):
    """Class defining a seasonal naive model that repeats the values of the past year as its prediction.

        Args:
            horizon: define the forecast horizon of the Naive model.
    """
    season: int = WEEKS_IN_YEAR
    horizon: int = HORIZON

    def predict(self, ts_signal: pd.Series) -> pd.Series:
        """ Compute prediction of a Naive model.

            Args:
                ts_signal: A pd.Series gathering a single time series signal.

            Returns:
                single_forecast: A pd.Series gathering the seasonal Naive prediction.

        """

        historical_data_past_year = ts_signal.tail(52)
        nb_repeat, part_repeat = divmod(52, 52)
        naive_forecast = [historical_data_past_year] * nb_repeat + [
            historical_data_past_year[:part_repeat]
        ]
        single_forecast = pd.concat(naive_forecast, axis=0, ignore_index=True)

        forecast_index = pd.date_range(
            start=ts_signal.index[-1], periods=52 + 1, freq=ts_signal.index.freq
        )[1:]
        single_forecast.index = forecast_index
        return single_forecast

Thus, here is a simple code base that we can use to compute a one-year prediction for Chunky Sneakers in 2024.

def load_time_series_csv(filepath: str) -> pd.DataFrame:
    df = pd.read_csv(filepath, index_col=0)
    df.index = pd.to_datetime(df.index)
    df.index.freq = df.index.inferred_freq
    return df


fashion_ts = load_time_series_csv('chunky_sneakers.csv')
fashion_ts_train = fashion_ts.loc[:"2023-01-01"]

ets_model = EtsForecastModel()
ets_prediction = pd.DataFrame(ets_model.fit_predict_single_ts(fashion_ts_train["chunky_sneakers"]))
snaive_model = SnaiveForecastModel()
snaive_prediction = pd.DataFrame(snaive_model.predict(fashion_ts_train["chunky_sneakers"]))

Here are the resulting predictions:

3) Forecast evaluation

When comparing the predictions with what actually happened in 2024, we can see that both models are quite accurate. The exponential smoothing model effectively captures seasonality and accurately predicts the increase in 2024, unlike the seasonal naïve model, which simply reproduces the previous year’s data. However, this increase appears to be slightly underestimated, as both models fail to fully match the growth at the end of 2023.

Several error metrics can be used to assess the performance of our two predictions and confirm what we observe in the graph above. Below, we provide the code for three of these metrics:

• Mean Absolute Scaled Error (MASE)

def mase(y_true: pd.DataFrame, y_pred: pd.DataFrame, seasonality: int) -> np.ndarray:
    """
    NOTE: y_true must contain at least `seasonality` time steps before the first time index
        of y_pred, plus the number of time steps in y_pred.
    """
    time_split = y_pred.index[0]
    index_y_pred = y_true.index.get_loc(time_split)
    assert index_y_pred > seasonality + 1, (
        f"To compute the MASE, you need {seasonality} time steps of history"
        f" before timestamp {time_split}."
    )

    numerator = np.mean(
        np.abs(y_true.values[index_y_pred:index_y_pred + len(y_pred)] - y_pred.values), axis=0
    )
    gt = y_true.values[seasonality:index_y_pred - 1]
    gt_shifted = y_true.values[:index_y_pred - 1 - seasonality]
    denominator = np.mean(np.abs(gt - gt_shifted), axis=0)
    return numerator / denominator

• Mean Absolute Percentage Error (MAPE)

def mape(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> np.ndarray:
    time_split = y_pred.index[0]
    index_y_pred = y_true.index.get_loc(time_split)

    numerator = y_true.values[index_y_pred:index_y_pred + len(y_pred)] - y_pred.values
    denominator = y_true.values[index_y_pred:index_y_pred + len(y_pred)]
    mape = np.abs(numerator / denominator)
    mape[mape == np.inf] = np.nan
    return np.nanmean(mape, axis=0)

• Symmetric Mean Absolute Percentage Error (SMAPE)

def smape(y_true: pd.DataFrame, y_pred: pd.DataFrame) -> np.ndarray:
    time_split = y_pred.index[0]
    index_y_pred = y_true.index.get_loc(time_split)

    numerator = np.abs(y_true.values[index_y_pred:index_y_pred + len(y_pred)] - y_pred.values)
    denominator = 0.5 * (
        np.abs(y_true.values[index_y_pred:index_y_pred + len(y_pred)]) + np.abs(y_pred.values)
    )
    smape = np.abs(numerator / denominator)
    smape[smape == np.inf] = np.nan
    return np.nanmean(smape, axis=0)

Now that we have these three metrics, we can compute the accuracy of the two predictions made in the previous part. All the values are small, confirming that both predictions are accurate, with the exponential smoothing model outperforming the seasonal naïve method.

4) Is fashion predictable?

Looking back at the previous sections, it seems we’ve our answer to the question: “is fashion predictable?”. By analysing social media and using simple statistical approaches, we can make accurate predictions of the future evolution of fashion.

However, the Chunky Sneakers trend remains relatively stable, with slow changes and clear seasonality. It doesn’t fully capture the complexity of the fashion world, where simple forecasting models may no longer suffice. In upcoming articles, we will continue our journey into fashion time series forecasting, working on new fashion time series, testing more powerful forecasting algorithms and assessing their performance.

If you’d like to replicate the graphs and results presented in this article, you can access the linked code repository here.

I leave you with a final graph displaying the evolution of Retro Football Sneakers in The United States along with the prediction from the exponential smoothing model for the year 2024. We’ll cover it in a later post !

Is fashion predictable? was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

There are currently two broads axis of research in Deep Learning: finding better architectures, or finding better losses to train them. Since AlexNet (Kriwhevsky et al.), Convolutional Neural Network is the main architecture used in Computer Vision. Through convolutions, with its efficient patch prior, many visual tasks were unlocked.

Many variations of AlexNet were invented, but if we were to name only one, it would be the ResNet (He et al.) and its residual shortcut.

The main drawback of convolutions is their lack of global reach. A problem, relatively similar to the difficulty of Recurent Neural Networks (e.g. RNN, LSTM, GRU) to model long series of tokens. The NLP community saw its “imagenet” moment when the Transformer (Vaswani et al.) architecture was unveiled. First designed with a somewhat complex encoder-decoder, it was later refined in a serie of identical blocks in the BERT (Devlin et al.) model. Using transfer learning, most (all?) tasks of NLP were attacked via a BERT-like architecture.

Two questions:

— What are exactly transformers?

— And can (should?) we apply them to Computer Vision?

1. What are Transformers?

I can only recommend the excellent blog post of Jay Alammar “The Illustrated Transformer“ which explains very well this architecture. But here is a quick recap, a transformer is basically that:

This whole block is repeated multiple times, the output of the first goes into the second and so on. Contrary to CNNs or MLPs, the number of dimensions doesn’t change across blocks.

You can also notice residuals twice. They are no different from the residuals in a ResNet, I guess that a trick so good it is found everywhere.

1.1. Tokens

The four red squares on the left are the tokens. There are only four on the image, but actually you can use as much as you (and your RAM) want. A token is simply an piece of data that was embedded. For example in NLP, you could have one token per word of your sentence. From this word, we can use its word2vec correspondence giving a latent vector. To also indicate where this word is located in the sentence, we add a positional encoding.

In this image, there is four tokens entering the block, and four tokens exiting the block. Therefore at the end of the N blocks, we will still have four tokens, but only one of them will be useful! The first token, here denoted by a ‘C’, is a special token called class token. It doesn’t belong to any word of the sentence, but is actually a vector that is learned during gradient descent. Once this class token has gone through the N blocks, it will be finally given to a classifier that will predict whatever you training for.

Put it simply, this class token will go through the network and will extract as much as possible useful information from the word tokens, in order to produce a good representation for the classifier.

1.2. Multi-Headed Self-Attention

This is the main part of the transformer. First, let’s concentrate on the ‘self-attention’.

We start with the matrix X of shape T x D containing all T tokens. This matrix is going to be linearly transformed (aka matrix multiplication, aka go to a fully-connected layer) three times in parallel:

They represent respectively the Query, the Key, and the V alue.

We compute the attention A, a square matrix of shape T x T:

As the name imply, this attention will determine how much a token correspond to another. Notice that the shape of attention matrix A is T x T. For each row i, the value of the column j is the strength of the relationship between the word i and word j.

We then multiply this attention matrix with the value matrix to select the data:

Where Z is a matrix which has the original shape T x D.

However, the attention matrix is quite restrictive: if the attention weight between tokens i and j is large, then because of the softmax it will be low between tokens i and u. In order to model multiple kinds of attention, we use multiple heads.

Let’s say we have two heads, then we do the previous self-attention but with different linear transformations:

But then, we have a matrix Z’ which is the concatenation of Z₁ and Z₂ and thus bigger than expected. We simply use again a linear transformation Wₒ of shape 2D x D:

Here is a simplified version in PyTorch:

Note that in practice, to make the dimension of Q/K/V independent to the number of heads, we usually fix the maximum embedding size and split it equally between heads some space.

1.3. Layer Norm and MLP

The rest of the transformer is made of a MLP and two layers norms. The former is simply several fully-connected layers with non-linear activations. The latter are a form of normalization layer where the input is subtracted by its mean and divided by its standard deviation, and then is linearly transformed by a set of weights and bias.

2. ViT: Vision Transformer

Contrary to convolutions and recurrent networks, transformers don’t really have a prior of the input data. So we should easily apply it on images, right?

Some have incorporated part of the self-attention, as Axial-DeepLab (Wang et al.). But none to the best of my knowledge, used the whole transformer architecture on images before ViT (Dosovitskiy et al.). Why? Well, in the case of images the number of tokens is huge. A 224 x 224 image has 50,176 pixels/tokens. This is too big to fit on a GPU.

The key idea, maybe inspired by BagNet (Brendel et al.), is to consider a group of pixels as a patch. Instead of thousands of tokens, the image is split into 14 x 14 = 196 tokens/patches, each made of 16 x 16 pixels.

The transformer architecture is exactly the same as one you could find in NLP. The only difference is how to embed the tokens. In this case, it’s simply a convolution applied patch per patch (called ‘linear projection’ in the image). In addition of positional encoding based on the position of the patch in the image is used.

The performances of ViT are very impressive. But a major drawback is that this architecture has almost no prior for images. Therefore, while it can potentially reach better results given infinitely large amount of data, it is very hard to train it with few images. The authors of ViT, all from Google, use JFT-300M a large private dataset to pretrain their model on it before transferring the knowledge to ImageNet-{21/1k}.

3. DeiT: A more efficient training

DeiT (Touvron et al.) is an extention of ViT with two main contributions: a more efficient training and a transformer-based knowledge distillation.

Through tons of regularization, the authors manage to train a DeiT from scratch on ImageNet with good performance. Two crucial regularizations are random erasing of the image pixels, and stochastic depth (Huang et al.). The latter is a kind of dropout, but where a whole transformer block is dropped with only the residual used.

The second contribution of DeiT is to propose to improve the training of the transformer by using a teacher model. The teacher/student strategy is often seen where a large trained teacher model produces novel targets to the student through knowledge distillation (Hinton et al.). Basically, the student is trained to predict the labels of the image but also to mimick the probabilities of the teacher.

The first interesting conclusion is that it’s better to use a large CNN (a RegNet (Radosavovic et al.)) as a teacher than a large vision transformer. Intuitively, we may hypothesize that is because a convolution-based teacher has learned different knowledge and thus may diversify better the predictions of the student.

The second conclusion is that using the output of the student classifier is not the best way to do knowledge distillation. Recall that the transformer’s classifier input is the class token that has gone through all blocks. DeiT instead add a new token, called a distillation token, and with it a new classifier. This classifier will be solely devoted to the task of distillation with the teacher, while the original classifier with class token will resume the image classification task.

3. ConViT: Make Prior Great Again

ViT needs a 300M dataset to train. DeiT only needs a 1.2M dataset (ImageNet) to train. But can’t we learn with a few thousand of images like a good old resNet can?

You need a prior to learn with few data point.

This is the message of ConViT (D’Ascoli et al.). They modify the usual Self-Attention (SA) into a Gated Positional Self-Attention (GPSA):

As shown on the diagram, only the attention part is modified, while the multiplication with the value matrix is left unchanged. This attention is now a linear combination between the self-attention and a positional prior. A learned gate (aka a learned vector followed by a sigmoid) balances between both branch, although its initialization enforces at first a strong emphasis on the positional prior. This positional prior, initialized the right way, can mimick a convolutional kernel.

A locality strength factor α can modulate how big this kernel should be:

The authors find that at the training end, the early transformer blocks (2–7) mainly use the positional prior, while the first and final blocks (8–10) use the global reach of the self-attention.

With few data, the positional prior has an important role as the following graph shows:

4. CaiT: Further refinements of DeiT

The authors from DeiT later released CaiT (Touvron et al.), a refinement of of DeiT.

The first contribution is LayerScale, a ponderation scheme for the main branch in the residual architecture. As for ResNet, each block has two branches: an identity branch that often does nothing, and the branch with learnable parameters. In the case of transformers, those parameters are either the MLP or the Multi-headed Self-Attention.

Previous works (ReZero Bachlenchner et al., SkipInit De et al., and FixUp Zhang et al.) wondered if we could train a very deep ResNet without batch normalization. They all found, with minor variations, that it was possible if main branch of a ResNet block was weighted by a learned scalar. LayerScale proposes more or less the same, but with one scale per dimension , initialized to a small ε value:

The second contribution of CaiT, is very similar to a contribution of ConViT (authors are partially the same): CaiT also introduces the class token after several blocks. The last few blocks with class tokens implement a Class Attention.

Contrary to Self-Attention, this Class Attention freezes the patch tokens:

Put it more simply, the patch tokens aren’t updated anymore after each block of Class Attention. Moreover, the attention is not done between all patches tokens Xp, of shape (T-1) x D, anymore (at a quadratic cost T²), but only with the class token Xc of shape 1xD (at a linear cost T):

The computational speed up is easily seen as the attention product is not anymore quadratic w.r.t the number of patches. Furthermore, it gives a nice performance gain, as the Self-Attention layers are now allowed to focus on extracting class-agnostic features.

5. The Future of Vision Transformers

ViT first proves that it was possible to train transformers on visual tasks. DeiT then showed that with carefully designed regularizations, the training could be done on relatively small scale datasets. This contribution, and the fantastic Timm library, opened a gold rush on transformers.

I won’t enumerate all variants here. Some may be only a slight tweak and will be forgotten, others may prove to be a stepping stone in the transformers world. Only time will tell.

But, that’s assuming that transformer is indeed the right solution. Attention is all we need? Recent research on MLPs (yes, you read correctly) indicates that maybe attention is only a special case of a more general framework.

We’ll cover it in a later post!

The Vision Transformers was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

• Fashion and music exist in mutual creativity, both acting as a zeitgeist for whatever the era’s trends and sentiments.
• Hip-hop in particular is enjoyed by a large portion of younger generations, and as a result fashion trends tend to crop up from the genre.
• Heuritech’s analysis of hip-hop music videos reveals the top fashion trends of 2019 and 2020, from Yo Gotti in fur to Ninho in checkers.

Fashion and music are inextricably linked. And they always have been, from Michael Jackson’s shiny glove to Pharrell’s vintage Vivienne Westwood hat. It’s not uncommon that a musical artist is the pioneer of a fashion trend, or even a fashion movement, it’s all about the universe the musician creates. And between Gen Z and millennials’ attachment to social media and penchant for the hip-hop genre, the interaction between fashion and music has never been so rapid fire. Cite Kanye West and his namesake brand Yeezy comes to mind; cite Tupac and one pictures a red bandana and gold chains.

These associations exist largely thanks to music videos, an art form in its modern form taking off in the 1980s with the rise of MTV. Today, music videos are diffused across social media to be watched on YouTube and other platforms by millions of fans, and with each album release it seems fans expect at least one or two videos to follow. Fashion brands haven’t let these numbers escape them: in Travis Scott’s music video for the song SICKO MODE, for example, he mentions the brand Nike, and has since been collaborating with the sportswear giant on their signature Air Force 1s. Moments like these speak to the power of images over fashion trends. So what are some of fashion’s major trends in hip-hop music videos in particular?

Fashion trends in the past year’s hip-hop music videos on YouTube

To answer this question, Heuritech analyzed over 600 of the most popular hip-hop music videos on YouTube across 2019 and 2020 in the US and France. Heuritech applied its in-house image recognition technology to do so, a method which allows for the detection of over 2000 attributes and products in each frame, like texture, color, model, and much more. Over the past year, five main trends stood out in some of hip-hop’s top music videos:

1. FUR

Of the music videos analyzed, 18% featured some sort of fur, most often worn on coats. It’s a classic look: big fur coat, ostentatious jewelry, hype entourage. In Dadju’s video for “Sous Controle” featuring Niska, the rapper walks into a mansion sporting a long fur coat before jumping into the song itself. In Yo Gotti’s video for “Put A Date On It” featuring Da Baby, Gotti stands at the head of a long dining table donning a red coat embellished with a fur collar. The fur coat is a symbol of beyond-rich, it’s why the look fits in so well with the hip-hop universe. Fur is a status symbol that’s stood the test of time, even in modern rap.

2. NEON & FLUORESCENT

From neon yellow to lime green, fluorescent colors were heavily featured in 2019 and 2020’s rap videos on YouTube with an 11% share. Featuring Nicki Minaj and Ty Dolla $ign, Megan Thee Stallion’s breakout video “Hot Girl Summer” is filled to the brim with girls in neon green, pink, blue, and more, with some of the set props even awash with fluorescence. Rich the Kid’s video for “Save That”includes children dressed in neon, dancing through a grocery store to the rapper’s music and even joining him to dance at times, equally dressed in a brilliant yellow. At times fluorescent colors tend to communicate a sort of futuristic, almost technological mood, while other times they evoke unbridled energy and enthusiasm. Rich the Kid and Megan Thee Stallion include neon colors as a way to transmit each mood through their music videos, respectively.

3. TANK TOPS

Call it what you may — singlet, string vest, undershirt, Guinea tee — men’s tank tops weren’t missed in hip-hop music videos. With a 10% share of the videos analyzed, this item was exclusively worn in black or white, typically with the rap-classic gold or silver chains to amp up the look. NLE Choppa’s “Shotta Flow Remix” featuring Blueface takes place at a backyard barbecue, where the rapper is dressed in the embodiment of summer relaxation: oversized jeans and a jersey tank top.

4. DENIM

Denim is one of those classic 90s and 2000s trends, from oversized jeans to denim trucker hats. This year, the trend has cycled back in smaller proportions: fitted skinny jeans, with rips or even acid wash, are a major hit. The trend appears in 17% of all music videos analyzed, which is no small number. In Kaza’s video for “Béni” featuring Maes, the rappers are filmed in front of suburban buildings, and Maes along with members of their entourage are sporting slim, ripped jeans with oversized jackets. K Camp’s music video for “Ice Cold” takes place in a 90s-style house, and again, the rapper wears ripped jeans with an oversized jacket.

5. CHECKERED

Checkers aren’t always an easy pattern to wear outside of prep school and 221b Baker Street, but this year’s hip-hop artists are breathing life into the preppy print. The pattern appeared in 11% of the music videos analyzed, often on flannel overshirts and jackets. Kpoint’s “Ma 6t a craqué” featuring Ninho, and DaBaby’s “Find My Way” are two videos in which both rappers appear in red and black checkered overshirts, à la Kurt Cobain. In Kodak Black’s “Calling My Spirit,” the women featured in the video sport black-and-white checkered jumpsuits.

Hip-hop meets high fashion

And when it comes to analyzing fashion trends, hip-hop music videos are a good place to look, particularly those posted to YouTube. In Nielsen Music’s 2020 mid-year report, the music sales tracking company reported that hip-hop is the top genre in the US and France this year. It’s not surprising given that the two countries are the top two producers of hip-hop in the world, which also helps to explain luxury brands’ interest in rappers and their videos. In the last two years alone, brands from Louis Vuitton to Saint Laurent have released rap-fronted campaigns to draw in younger customers. “It is from rap and music video that followers are able to determine and validate their assumptions about their lifestyle decisions, including apparel expressions,” states Cornell Professor Van Dyk Lewis.

And the list of collaborations between rappers and designers thanks to their lyrics and music videos is seemingly exhaustive, dating back to even the 80s. Run-DMC’s 1986 song My Adidas scored the group a million-dollar endorsement, a deal unheard of at the time. Since, the list has only grown, including Versace collabing with 2 Chainz on a sneaker, Dior selecting S.Pri Noir as the face of a fragrance campaign, and Chanel partnering with Pharrell on a 2019 collection. Consumers see these collaborations, and they want what they see. Genius notes that in 2018, “the three most-referenced fashion brands–Fendi, Balenciaga, and Gucci–were called out 664 times in rap and R&B songs.” While fashion and music is no one-way street, can anyone be surprised that these brands are some of the most-desired by younger consumers today?

The power of music videos in hip-hop’s fashion trends

Fashion and music exist in mutual creativity, and it’s clear that their expression is often appreciated via music video. These videos seem frozen in time when one looks back, and in a way they are fashion’s living archives.

When Ja Rule wore a rare Burberry bucket hat in his music video for “Always on Time,” many fans of hip-hop discovered the famous tartan pattern for the first time, and the brand has been present in rappers’ outfits ever since. When A$AP Rocky wore and mentioned Dior in more than a few of his music videos, he became a face of Dior Homme shortly thereafter in 2016. When PNL appeared in custom Off-White in their music video for “Au DD,” a capsule collection between the French rap duo and Virgil Abloh was announced not long after.

There are seemingly endless instances of rappers mixing with high fashion, and more and more, it seems to go the other way around, with brands instead approaching rappers. The reach of these artists on fashion trends is further backed by Heuritech’s analysis of the past year’s top five trends drawn from hip-hop’s top music videos on YouTube: fur, neon, tank tops, denim, and checkered.

The year’s top fashion trends in hip-hop, as seen on YouTube was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

• Brands are eager to find the best way to be present on TikTok, in the hopes of engaging with the platform’s young user base
• High potential for user engagement seems to be the key for success on the platform, with tutorial videos and challenges being amongst the most popular video formats
• TikTok communities have influenced style, with Celine’s SS21 menswear collection titled “The Dancing Kid” being one of the latest examples of this phenomenon

This summer, a number of young TikTok stars including Chase Hudson (@lilhuddy) and Noen Eubanks (@noeneubanks) served as the face of Celine, with creative director Hedi Slimane’s SS21 Gen Z-inspired collection titled “Dancing Kid”, solidifying TikTok’s vital role in the fashion ecosystem. The video sharing platform is particularly popular amongst Gen Z, who represent the largest segment of its user base. Thus, it comes as no surprise that luxury brands are eager to figure out how to effectively engage with TikTok users in the hopes of attracting Gen Z to their brands.

Virality on TikTok

However, going viral on TikTok is a major challenge for even the biggest brands, as the type of content that is popular on Instagram often does not perform well on TikTok. This is partially because, compared to Instagram, TikTok is described as being more “authentic” and “spontaneous,”with the opportunity for user engagement being an important aspect of a video’s success.

Popular video formats on TikTok include various “challenges”, educational videos, and, of course, lip syncing and dancing. Since the popularity of these video formats rely on the mass participation of the TikTok community, with users recreating their version of a viral dance, for instance, it is clear that a high potential for user engagement seems to be the key for success on the platform.

Another idiosyncrasy of TikTok compared to other social media is that the content itself is ostensibly more important than the user creating it. While there are clearly anomalies to this tendency, as TikTok influencers do exist (take the Hype House, for one), it is much more difficult for fashion brands or celebrities to be successful on the platform simply because they are well known. According to an article published by Business of Fashion, the best way for brands to benefit from TikTok is “either by inventing a meme that catches on with users or, more often than not, finding a way to capitalise on content that went viral organically.”

Fashion brands trending on TikTok

With this in mind, it makes sense why brands are struggling to be successful on the platform, as engineering a viral moment is difficult, and having one organically appear is pure luck. Gucci is the latest luxury brand to benefit from a viral TikTok moment, with the #GucciModelChallenge where TikTokers emulate the brand’s signature kitschy aesthetic by creating looks from their own wardrobe. At the time of writing this article, this hashtag has generated over 14.5 million views despite Gucci having zero involvement in creating the challenge.

The “democratic” nature of the platform, where seemingly anyone can become famous if their content resonates with fellow TikTok users, is exactly what makes the app so appealing. For instance, a JW Anderson patchwork crochet cardigan went viral after Harry Styles was seen wearing it during a rehearsal on The Today Show. Being priced at 1450€, far beyond the budget of most Gen Zers, one TikTok user, Liv Huffman (@lilbittylivie) created her own version of the cardigan in a video that reached 3.6 million views, and inspired thousands of other users to recreate their version of the garment. Following this explosion on TikTok, JW Anderson even released the pattern for the cardigan to encourage others to participate in this DIY trend.

This example is emblematic of the type of content that is well suited to TikTok — according to a recent article by Vogue Business, “Whenever you’re teaching people something, your engagement is higher,” which is logical as users are inspired to recreate the content of the video. The prominent DIY community on TikTok is related to the fact that Gen Zers tend to be more environmentally conscious than previous generations, and appreciate hand-made garments.

Moreover, due to the pandemic, people are spending more time at home and on their devices, with more time to develop hobbies, making the DIY trend particularly relevant for 2020. In fact, the term “craftcore” has recently emerged, describing this hand-made aesthetic and practice, with tie dye, patchwork, and crochet being some stylistic elements of this new subculture.

Despite the environmental awareness of Gen Z, it is important to mention that fast fashion brands like Boohoo or PrettyLittleThing are still finding success on the platform. Boohoo created a TikTok song along with a challenge, tagged #Itsfromboohoo generating 122.7 million views, while PLT’s #prettychallenge has 34.8 million.

Trending subcultures in fashion on the platform

The dialogue between fashion and TikTok is informed by the distinctive subcultures that emerge on the platform. For example, Celine’s most recent men’s collection was inspired by the “eboy” aesthetic that describes a specific archetype of Gen Z, with chunky chains, oversized cardigans, and clashing patterns being the token elements of the style.

Although it is unclear where subcultures like the eboy/egirl or craftcore begin, TikTok certainly nurtures the development of these communities, making the platform an indispensable form of social media in the fashion industry. TikTok presents a change to the “influencer industry”, because brands and celebrities cannot rely solely on their fame to succeed on the platform. Rather, they must create engaging content, like challenges or tutorial style videos that encourage user interaction, in order to be popular on TikTok.

The future of TikTok fashion trends

Although brand exposure on TikTok sometimes arises by chance, with the #GucciModelChallenge or JW Anderson sweater, it is clear that if fashion brands want their content to go viral on the platform they need to invent new ways to encourage users to engage with their content.

This is where data drawn from the platform can be highly valuable to brands seeking to peer into the daily behavior of its users: Heuritech analyzes images and videos shared on social media in order to historicize and predict fashion trends. Because trends evolve so rapidly on TikTok, it is key to be able to assess thousands, even millions, of images per day. Heuritech currently analyzes over 3 million images per day on Instagram alone — the possibilities for TikTok are good news for luxury brands seeking to be first string in fashion trends. In any case, TikTok’s influence in fashion attests to the importance of social media in predicting future trends, offering a snapshot of consumer behaviours and preferences in the virtual realm.

How Can Fashion Brands Go Viral on TikTok in 2020? was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.

• Men’s SS21 digital fashion week represents a turning point in the fashion world, being the first major digital fashion week following Haute Couture, with designers creating innovative new ways of presenting their collections, challenging the traditional format of the fashion show
• Following this period of crisis, brands are questioning their reason to exist, with the meaning and intention behind their collections being equally as important as the garments themselves
• A major theme this fashion week was a “return to nature”, with designers showing a renewed appreciation for the natural environment, and favouring more artisanal materials and textiles that require a high level of craftsmanship

This men’s SS21 digital fashion week was a pivotal moment for the fashion world, with both independent designers and luxury maisons being tasked with the challenge of somehow translating the experience of attending a show IRL to the virtual realm. Although there is no true replacement to the “real thing”, many designers took advantage of this unique opportunity to challenge the format of a traditional fashion show, and utilize the possibilities afforded by technology to communicate with their audience in a novel way. Another first related to this (almost) exclusively digital fashion week is the democratization of the viewing experience, as would-be fashion show attendees and the “average” person both see the collection through the screen of their devices.

Brands were forced to adapt to this new reality, with many coming up with highly innovative ways of presenting their collections in the digital realm. UNDERCOVER’s collection was completely rendered in 3D, with models appearing like avatars from Grand Theft Auto; Sunnei’s “Canvas” collection utilizes CGI to depict models wearing white garments that are completely customisable through the brand’s VR platform; COOL™ created an interactive experience using VR; Kolor’s Junichi Abe developed a 360 degree camera system so viewers can see each look from every angle.

Narration to Convey Emotion

In addition to experimenting with the format of the digital fashion show, several designers took a philosophical approach in attempting to convey a deeper meaning behind the collection that goes beyond the physical garments. A distinctive theme that emerged this fashion week is using narration to convey a sense of emotion, with designers’ videos featuring poetry, novel excerpts, home-video audio recordings, and Nietzsche quotes dubbed over visuals. The Bolognese brand Magliano created a mythological universe, with each model embodying a fictitious character, featured in a video narrated by the novelist Isabella Santacroce. Luchino Magliano, founder of his eponymous label, explains that “Narration is the first form of augmented reality. We asked ourselves how we could overcome the virtual medium to convey the emotion and research behind each object, and we came to the conclusion that the word is the strongest medium.” Given the amount of designers who utilized narration to convey the story behind their collections, it is clear that many others would agree with Magliano.

Return to nature

Ironically, a prevailing theme this fashion week was a “return to nature”, despite collections only being presented in the digital realm. Numerous brands either focused on nature as a guiding theme, or incorporated the natural environment into their collections in some capacity, whether it be floral motifs, lookbooks shot amongst vegetation, or plants physically attached to garments. This gravitation towards nature is ostensibly related to the desire to convey a deeper meaning and a sense of emotion with viewers via the internet, as times of crisis often result in a renewed appreciation for beauty, especially after being locked indoors for several months.

Bohemian Inspiration

A key inspiration for this back to nature theme is the 60s and 70s, which is logical, since the aesthetics of these epochs are defined by political upheaval and a questioning of the established order. Since we are experiencing both of these phenomena today, this free-spirited, bohemian sensibility ostensibly resonates more strongly than it would otherwise, manifesting through the physical garments in the form of paisley, patchwork, ikat patterns, fringe, tinted aviators, and loose, flowing silhouettes, shown by brands like Children of the Discordance, Isabel Marant, Greg Lauren, Etro, and Magliano. Tunics, knitwear, and espadrilles were also token elements of this theme, shown by the brands that were inspired by this free-spirited, bohemian mood.

Hippy Streetwear

Another iteration of this natural theme comes from a more streetwear perspective, with brands such as Blue Marble, Etudes, and MSGM combining t-shirts, hoodies, as well as more tailored garments with the typical “hippy” accoutrements, like natural-dyed textiles, beads, rope accessories, psychedelic prints, and vibrant colours paired with muted, “sun-drenched” tones. These details emanate a sense of joy and optimism, with vibrant colours and hallucinogenic motifs symbolising a type of rebirth, as we are entering a new phase of life following this period of confinement. Massimo Giorgetti, creative director of MSGM, states that “for the first time in my life and in the life of the brand, I perceived the fall of the myth of the city as the place to be and the rise of a new desire for nature”. Clearly, Giorgetti is not alone in this desire to be connected to our natural world and have a fresh start.

Focus on Craftsmanship

Lastly, a key trend was a gravitation towards artisanal materials and techniques, with an abundance of embellishment, knits, linen, woven textiles, and embroidery representing a renewed appreciation for the craftsmanship required to create such garments. CMMN SWDN, Ermenegildo Zegna, and Jacquemus are a few examples of those who incorporated these artisanal elements into their collections, combining these “crafty” details with tailoring to create a level of sophistication. Some staple garments amongst brands that explored this theme include espadrilles, tunics, and loose fitting pants, typically in earthy, muted tones.

Ultimately, this fashion week seemed to predict, at least in part, a return to a time where collections were smaller, craftsmanship was rightfully given the spotlight, and nature was directly reflected in the product. In other ways, tradition was challenged, with the digitalisation of many catwalks and look books giving access to more than only industry insiders. What we do know is that these trends inspire us to keep our roots in the ground and our eyes to the sky.

Heuritech empowers fashion teams to monitor in real time both products and trends through cutting-edge image recognition technology applied on millions of social media images every day. If you want to know more, get in touch at info@heuritech.com

Quarantine Rebirth: Men’s Fashion Week Review SS21 was originally published in Heuritech, a Luxurynsight Company on Medium, where people are continuing the conversation by highlighting and responding to this story.