On September 18, Medium held Medium Day 2026, its yearly celebration of writing and ideas. It was a free, day-long event on Zoom with around a hundred sessions. I co-hosted one of them, AI: Hype, Reality & Writing, with Grace Gong and Mark Dranias.

My ten minutes were about cognitive offloading: what happens to our thinking when we hand more of it to AI.

The pair of numbers

I built my part around one study. In a Wharton-led experiment published in PNAS, nearly a thousand high-school students practised maths with a chatbot. Their practice scores rose 48%. Then the chatbot was taken away, and they scored 17% worse than students who had never used it.

A second group got a version that gave hints instead of answers. They kept the gain and avoided the drop. Same model, same students, opposite result. The difference wasn’t how much AI they used. It was whether they tried first.

What the room said

I asked everyone: What’s one thing you’ve stopped doing yourself since AI?

The biggest answers were coding, summarising, research and proofreading. “Nothing” was just as big. And scattered around them were “thinking,” “overthinking” and “thinking in my own head.” Almost nothing on the screen was typing work. Nearly all of it was thinking work.

What I noticed

Everyone is using AI. In one short hour I met 150+ people from tech, linguistics, medicine, business, education, ecology and the human sciences. This isn’t just a conversation among tech folks anymore.

Everyone is worried about reliability. Are the guardrails enough? Do we need humans in the loop? Underneath all of it was one question: how do we decide when to trust it?

The environmental cost barely comes up. We talk a lot about what AI does to our work, and very little about its carbon footprint or what it does to the world around us.

The fear has turned practical. People love the accessibility AI brings. The dread of the unknown has mostly given way to a working relationship.

What I’m taking home

The rule I offered the room is the one I’m keeping for myself: delegate the output, not the first attempt. If a piece of writing is just a deliverable, let AI help. If I’m writing to find out what I think, the first draft has to be mine.

I closed the session with a question, and I’ll leave it with you too: When did you last write something and not know what you thought until you’d finished?

Thank you to Grace, Mark, everyone who joined, and the Medium team for having us.

There’s an older, more formal discipline for exactly this problem: ontologies, from the semantic web world. They sound academic, and the tooling around them (OWL, RDF, SPARQL, Protege) can feel like overkill for a side project. But the core idea — write down your domain’s vocabulary before you build the database, in a form a machine can actually check — is one of the more underrated moves you can make before designing a graph schema. This post walks through what an ontology actually is, how to build a small one, and how it translates cleanly into a Neo4j property graph.

What an ontology actually is

An ontology is a formal specification of a domain: the categories of things that exist in it (classes), how they relate to each other (properties), and the constraints those relationships obey. It’s built on two W3C standards:

• RDF (Resource Description Framework) — everything is a triple: subject–predicate–object. Book1 – writtenBy – Author7.
• OWL (Web Ontology Language) — a layer on top of RDF for declaring classes, property types, domains/ranges, and logical characteristics (symmetric, transitive, functional, and so on).

Two halves matter in practice:

• TBox (terminological box) — the schema. “A Book can be writtenBy an Author." Defined once, changes rarely.
• ABox (assertional box) — the instances. “The Left Hand of Darkness is a Book, writtenBy Ursula K. Le Guin." Grows with every record you load.

This is the same split as “schema vs. data” in any database — the difference is that OWL gives the schema itself a formal, machine-checkable meaning, not just a set of table definitions you have to remember the intent of.

Why bother, concretely

1. The vocabulary is explicit and typed. writtenBy isn't a column name you have to remember the semantics of — it's a declared property with a domain (Book) and range (Author) that any tool can introspect.
2. It’s checkable, not just documented. Run an OWL reasoner over the graph and it can catch a genuine inconsistency — like an edge that violates a declared domain/range — instead of that only being a comment in a wiki nobody reads.
3. It’s queryable independent of your database. A .ttl (Turtle) file loads into Protege, answers SPARQL queries, and can be handed to someone with zero other context.
4. It decouples “what the domain looks like” from “how I happen to store it.” The ontology doesn’t know or care that you picked Neo4j. Swap the database later and the TBox doesn’t change.

Building a small TBox

Let’s use a concrete, deliberately small domain: a book catalog. Not e-commerce, not movies (Neo4j’s own tutorials have that one covered) — just books, the people who write them, who publishes them, and what they’re about.

A few deliberate choices here, worth calling out because they’re the kind of decision an ontology forces you to make explicitly:

• Book subclasses CreativeWork. If this catalog ever grows to include audiobooks or short-story collections, they inherit the same base properties (title, publishedDate) without redefining them.
• Author subclasses Person, not the other way around — because an author is a kind of person, and future classes like Translatoror Illustrator can subclass Person too without duplicating properties like name or birthDate.
• Publisher and Genre stay flat, siblings of nothing. Resist the urge to build hierarchy you don't have evidence for yet — a subclass relationship is a commitment. Real books-domain hierarchies (e.g. Genre having sub-genres) are usually better modeled as an object property (Genre –broaderThan→ Genre) than a rigid class hierarchy, because genre taxonomies change and overlap in ways class trees don't handle gracefully.

Object properties: how instances relate

Object properties connect one instance to another. Each has a domain (where it starts) and range (what it points to) — this is what lets a reasoner flag a mistake like linking a Genre to a Publisher.

PropertyDomainRangeNoteswrittenByBookAuthorpublishedByBookPublisherhasGenreBookGenrepartOfSeriesBookSeriesprecedesBookBookordering within a seriessimilarToBookBooksymmetric — see below

That last property is worth a closer look. Declared as owl:SymmetricProperty, similarTo gives you a real inference for free: if A is similar to B, B is similar to A, without your application code ever having to enforce it. That's the kind of guarantee that's easy to forget to maintain by hand once you have a few thousand rows, and trivial to get right when it's declared once in the schema.

Datatype properties: the actual values

Where object properties link two instances, datatype properties attach a literal value — a string, a date, a number — to one instance: title, isbn, publishedDate, pageCount, averageRating, language.

Two worth a second look, because they solve a real, recurring problem rather than just describing a field:

• externalId + sourceSystem (if you're pulling data from more than one API) exist specifically for entity resolution. If a book comes from both an ISBN registry and a reviews API, these two properties let you represent "this instance's ID in system X is Y" as a first-class, queryable fact, rather than burying that mapping in an ad-hoc join table you have to remember exists.
• A single title field is usually wrong. If your domain has any multilingual or multi-edition angle, model titleOriginal / titleLocalized as separate typed properties from the start — which one you want depends on context (search vs. display vs. matching against a source-language dataset), and retrofitting that split later means a migration, not a one-line addition.

From TBox to ABox: populating instances

The TBox is static. The ABox is where your actual data lives — and the interesting design decision here is how you mint identifiers.

The detail that matters most in practice: use deterministic URIs, not random ones. A book gets data#book/{isbn}; an author gets data#author/{slugified-name}. Because RDF triples are set members, re-asserting the same triple twice is a no-op — so re-running your loader against data you've already ingested doesn't create duplicates, it just re-confirms facts that are already true. That's idempotency at the triple-store level, and it's the exact same property you want from your database loader.

Translating the TBox into a Neo4j schema

This is the payoff: once the ontology exists, the Neo4j schema isn’t a separate design exercise — it’s a disciplined, near-mechanical translation.

• Every OWL class becomes a node label: Book, Author, Publisher, Genre, Series.
• Every OWL object property becomes a relationship type, conventionally upper-snake-cased: writtenBy → WRITTEN_BY, hasGenre → HAS_GENRE, similarTo → SIMILAR_TO.
• Every OWL datatype property becomes a node property in Cypher’s property map.

One real divergence between the two models, and it’s worth knowing about before it surprises you: RDF triples don’t natively carry properties on the edge itself — a plain triple is just subject–predicate–object. Neo4j’s property graph model, by contrast, lets you attach attributes directly to a relationship (SIMILAR_TO {score: 0.82}). In practice this means the ontology stays the formal source of truth for vocabulary — what relationships exist and what they connect — while the property graph is where you put per-edge metadata like scores, ranks, or timestamps that don't cleanly fit the pure-triple model. (RDF has ways to represent edge properties too — reification, RDF-star, singleton properties — but for a schema whose end target is Neo4j, it's usually simpler to just let the property graph own that part.)

And the loader should use the same idempotency principle as the ABox builder, just expressed as MERGE on a natural key instead of a deterministic URI:

MERGE (b:Book {isbn: $isbn})
SET b.title = $title, b.published_date = $published_date, b.page_count = $page_count
MERGE (a:Author {slug: $author_slug})
SET a.name = $author_name
MERGE (b)-[:WRITTEN_BY]->(a)
MERGE (g:Genre {name: $genre_name})
MERGE (b)-[:HAS_GENRE]->(g)

Never CREATE. MERGE-on-natural-key means you can re-run an ingestion pipeline against the same source data as many times as you want and the graph doesn't grow duplicates or drift — which matters a lot the moment you're resuming a multi-stage pipeline from the middle instead of starting clean every time.

What this buys you, concretely

• A domain model you can actually validate. Run an OWL-RL reasoner over the TBox+ABox and a real inconsistency — a Genre node with a WRITTEN_BY-shaped edge, say — fails the check. That's a sanity pass you get for free instead of hand-writing custom validation.
• A second query surface. The Turtle file answers SPARQL independent of Cypher — handy for ad hoc questions, or for handing the schema to a tool or team that doesn’t touch your database at all.
• A schema that’s already ahead of your data. If you define Series and Translator in the TBox before you have any translated or serialized books in the catalog, adding that ingestion path later is additive — the class is already there, waiting, not a redesign.
• One vocabulary across every layer. Because the graph schema is a disciplined mapping from the ontology instead of an independently-invented one, WRITTEN_BY means the same thing in the ontology docs, the Cypher queries, and — if you ever expose one — the API. Nobody has to guess whether it points from book to author or the other way around, because it was declared once, formally, at the start.

When you can skip this

Worth saying plainly: this is not free, and it’s not always worth it. For a weekend prototype, a single-person project you’ll rewrite in a month, or a graph with three node types and no ambition to grow, writing a formal OWL ontology first is very likely wasted ceremony — just design the Cypher schema directly and move on.

The formality earns its keep when at least one of these is true: the domain is genuinely complex and will keep growing, multiple people (or tools) need to share one unambiguous vocabulary, the data comes from more than one source and needs entity resolution, or you actually want the “can a machine check this for consistency” property rather than just a schema you trust yourself to remember correctly.

If none of those apply yet, skip the ontology and revisit it once they do — the TBox/ABox split means you can always formalize a schema you’re already running, rather than needing to have started there.

That’s all from me today! Fascinating topic, isn’t it? Hopefully this helps! Until next time!

If you’ve just started learning about machine learning, you’ve probably heard this phrase a hundred times: “garbage in, garbage out.” It sounds obvious, but the reality of making ML systems work in the real world — what we call production — is way more nuanced than just cleaning up a CSV file and calling it a day.

This article is part of a broader topic called MLOps (Machine Learning Operations). As Chip Huyen explains in her MLOps guide¹, “Ops in MLOps comes from DevOps, short for Developments and Operations. To operationalize something means to bring it into production, which includes deploying, monitoring, and maintaining it.”

Today, we’re zooming in on one of the most critical pieces of that pipeline: Feature Engineering.

Before we begin, here’s a visual summary of the article.

First, Why Does Data Quality Matter So Much?

Before we dive into the techniques, here’s something important to keep in mind:

More data of low quality can actually hurt your results. What truly matters is good quality and large quantity of data.

This is a really common trap for beginners — assuming that dumping more data into a model will automatically make it smarter. It won’t. A thousand noisy, mislabeled examples can be far more damaging than a hundred clean, well-labeled ones.

What Even Is Feature Engineering?

Features are the inputs you feed into your ML model. Feature engineering is the process of taking raw data and transforming it into a format that your model can actually learn from effectively.

There’s an interesting debate in the ML world right now: feature engineering vs. feature learning. The promise of deep learning was that models would automatically learn the best features from raw data — no manual work needed. And while that’s partly true for things like images and text, we’re still far from that reality for most real-world tabular data problems. So yes, you still need to understand feature engineering. 😄

Common Feature Engineering Operations

Here’s a rundown of the most important operations you’ll encounter, drawn from Chip Huyen’s lecture notes on Feature Engineering¹:

1. Handling Missing Values

Real-world data is messy. Sensors fail, users skip form fields, systems crash. You’ll almost always have gaps in your data that need to be addressed before training.

2. Deletion

Sometimes the simplest fix is to drop rows or columns with too many missing values — but be careful, this can introduce bias if the missingness isn’t random.

3. Imputation

Instead of deleting, you fill in the gaps. Common strategies include using the mean, median, or a model-predicted value.

4. Scaling

ML algorithms are often sensitive to the range of your features. A feature ranging from 0–1 and another ranging from 0–1,000,000 can confuse your model. Scaling (like normalization or standardization) brings everything to a comparable range.

5. Discretization

Converting continuous numerical values into discrete buckets or categories. For example, converting “age” into groups like “18–25”, “26–35”, etc.

6. Encoding Categorical Features

Models need numbers, not text. If your data has categories like “red”, “blue”, “green” — you need to encode them numerically. A popular technique here is hashing, which is especially useful when you have a very large number of categories.

7. Feature Crossing

A powerful technique where you combine two or more features to create a new one. For example, combining “city” and “time of day” into a single feature might help a model learn patterns it couldn’t pick up from either feature alone.

8. Positional Encodings (Discrete & Continuous)

Particularly important in NLP (Natural Language Processing). Since models don’t inherently understand word order, positional encodings inject information about where each word sits in a sentence. Under the hood, this uses an embedding matrix.

Data Augmentation: Getting More From What You Have

Another key technique in feature engineering is data augmentation — a way to artificially increase the size of your training dataset without collecting new data. There are three main flavours¹:

1. Simple Label-Preserving Augmentation Replace or add something with a similar meaning. In NLP, this might mean swapping a word for its synonym. The label stays the same, but you’ve created a new training example.
2. Perturbation Intentionally add noise to your data — a label-preserving operation used to make models more robust, teaching them to handle imperfect, real-world inputs.
3. Data Synthesis Generate entirely new data points that didn’t exist in your original dataset. Think of techniques like GANs (Generative Adversarial Networks) for images.

Sampling: Picking the Right Subset of Data

Before you start engineering features, you need to think about which data you’re working with. This is called sampling, and doing it wrong can introduce serious bias into your model¹.

Non-Probability Sampling

Selection isn’t based on any statistical probability:

• Convenience sampling — use whatever data is available
• Snowball sampling — future samples are selected based on existing ones
• Judgement sampling — an expert decides what to include
• Quota sampling — manually set proportions (e.g. 50% of group A, 20% of group B)

These methods are quick, but they risk producing samples that don’t represent the real world.

Random (Probability) Sampling

• Simple random sampling — every data point has an equal chance of being selected, but rare categories might not appear
• Stratified sampling — divide your data into groups first, then sample from each; much better for imbalanced datasets
• Weighted sampling — assign weights to samples rather than groups
• Importance sampling — useful when you only have access to one distribution but need to sample from another; common in reinforcement learning
• Reservoir sampling — a clever algorithm for sampling from a data stream when you don’t know the total size upfront

Bringing It All Together: The Feature Store

In production MLOps systems, managing features across multiple models and teams gets complicated fast. That’s where the Feature Store comes in — a centralized repository where you standardize the definition, storage, and access of features for both training and serving predictions¹.

This concept fits neatly into what Google Cloud describes as MLOps Level 1, where the goal is ML pipeline automation². As they explain, a mature MLOps system introduces automated data validation, automated model validation, automated pipeline triggers, and automated metadata management² — all of which depend on having clean, well-managed features to work with.

Key Takeaways

• Good data quality matters more than data quantity alone
• Feature engineering is still very much a human-driven process — don’t wait for models to do it all for you
• There are many techniques at your disposal: imputation, scaling, encoding, feature crossing, and more
• Sampling strategy matters — a biased sample leads to a biased model
• In production systems, centralizing features in a Feature Store saves time and prevents inconsistencies

Happy engineering! 🚀

References

¹ Huyen, C. (2022). MLOps Guide. huyenchip.com/mlops — includes lecture notes on Feature Engineering and Training Data (originally developed for Stanford CS329S: Machine Learning Systems in Production)

² Google Cloud. (2024). MLOps: Continuous delivery and automation pipelines in machine learning. Google Cloud Architecture Center. cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning

Keeping track of these experiments manually can quickly become chaotic.

This is where MLflow comes in.

MLflow is an open-source platform designed to manage the entire machine learning lifecycle, including experiment tracking, model packaging, and deployment.

In this article, we’ll walk through the basics of MLflow, understand its core components, and run a simple example to track a machine learning experiment.

Why Experiment Tracking Matters

When developing machine learning models, it is common to run dozens or even hundreds of experiments.

For example, you might:

• Test different algorithms
• Tune hyperparameters
• Experiment with different datasets
• Modify preprocessing pipelines

Without proper tracking, it becomes difficult to answer simple questions like:

• Which experiment produced the best model?
• What hyperparameters were used?
• Which dataset version was used?

Tools like MLflow solve this problem by automatically logging experiment metadata, metrics, and artifacts in an organized way.

What is MLflow?

MLflow is an open-source platform developed by Databricks that helps manage machine learning workflows.

It supports multiple ML frameworks including:

• Scikit-learn
• TensorFlow
• PyTorch
• XGBoost

MLflow is framework-agnostic, which makes it easy to integrate into existing ML pipelines.

MLflow Architecture

At a high level, MLflow consists of several components that interact with a central tracking server.

The key components include:

Experiment Tracking

Tracks key information during training such as:

• parameters
• metrics
• artifacts

This allows easy comparison between different runs.

MLflow Projects

MLflow Projects package machine learning code into reproducible runs.

They typically include:

• source code
• environment configuration
• dependencies

This ensures that experiments can be reproduced across machines.

MLflow Models

MLflow provides a standard format for packaging machine learning models so they can be deployed consistently.

Models can be served through:

• REST APIs
• batch inference pipelines
• production services

Model Registry

The Model Registry helps manage different versions of models and supports lifecycle stages such as:

• Staging
• Production
• Archived

This is especially useful for production ML systems where multiple model versions need to be managed.

Installing MLflow

MLflow can be installed easily using pip:

pip install mlflow

Once installed, you can start the MLflow UI with:

mlflow ui

This launches the tracking dashboard locally at:

http://127.0.0.1:5000

Running a Simple Experiment

Let’s run a simple example using Scikit-learn.

We will train a Logistic Regression model and log its parameters and accuracy using MLflow.

import mlflow
import mlflow.sklearn

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

X, y = load_iris(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(X, y)

with mlflow.start_run():

    model = LogisticRegression()
    model.fit(X_train, y_train)

    accuracy = model.score(X_test, y_test)

    mlflow.log_param("model_type", "LogisticRegression")
    mlflow.log_metric("accuracy", accuracy)

    mlflow.sklearn.log_model(model, "model")

print("Run logged successfully")

When this script is executed, MLflow automatically records the experiment.

Viewing Experiments in the MLflow UI

Once the run is completed, the experiment appears in the MLflow dashboard.

The dashboard displays:

• logged parameters
• performance metrics
• associated model artifacts

In this run, the model achieved an accuracy of approximately 0.92, and MLflow logged both the metric and the model type parameter.

Tracking Models

MLflow also stores the trained model artifact.

From the Models section, you can:

• view stored models
• access artifacts from experiments
• manage model versions

This makes it easier to transition models from experimentation to deployment.

Benefits of Using MLflow

MLflow simplifies many aspects of machine learning development:

• Organized experiment tracking
• Reproducible training workflows
• Framework-agnostic model management
• Easier collaboration within teams

It is especially valuable when machine learning projects grow beyond a few simple experiments.

Final Thoughts

Managing machine learning experiments effectively is a key part of building reliable ML systems.

Tools like MLflow make it easier to track experiments, compare models, and manage the lifecycle of machine learning models.

Even simple projects benefit from experiment tracking. Starting with tools like MLflow early in your workflow can make scaling and collaboration significantly easier.

If you’re building machine learning pipelines, integrating MLflow into your workflow is a great step toward production-ready ML development.

We are living through a strange moment in AI.

You can fine-tune a large language model without understanding backpropagation.
You can deploy a multimodal system without writing a single optimization loop.
You can build an AI product without ever seeing a gradient.

The abstraction layers are powerful — and seductive.

But beneath every Transformer, every diffusion model, every reinforcement learning agent, there is still a simple engine running:

Gradient descent.

And that hasn’t changed.

The Illusion of Intelligence

Large language models feel intelligent.

They generate essays, reason about math, write code, analyze images, summarize research papers.

But during training, none of that exists.

There is no reasoning.
There is no intention.
There is no understanding.

There is only:

A prediction

An error

A gradient

A small update

Repeated billions of times.

When you strip away the architecture diagrams and benchmark charts, modern AI is still iterative optimization.

Understanding that changes everything.

Scale Has Increased. Principles Have Not.

Today’s models operate at unprecedented scale:

• Billions to trillions of parameters
• Massive distributed training systems
• Carefully engineered learning rate schedules
• Precision-aware optimizers

But the update rule is still conceptually:

θ = θ - α ∇J(θ)

The sophistication lies in:

• How we compute gradients efficiently
• How we stabilize training
• How we structure the objective

But the core mechanism remains untouched.

And if you do research in AI — especially in high-stakes domains like healthcare, sustainability, or safety — this matters deeply.

Because when training becomes unstable, abstraction collapses.

You cannot debug trillion-parameter systems if you do not understand slope-following at a fundamental level.

Why Writing Gradient Descent From Scratch Changes Your Thinking

At some point in my own work, I stripped away every framework and implemented gradient descent manually.

Not because I needed to.
But because I wanted to feel the mechanism again.

Watching the loss decrease iteration by iteration forces a realization:

The model is not “learning concepts.”
It is navigating geometry.

It is moving across a high-dimensional surface, searching for a valley.

When you understand that, several things become clearer:

• Vanishing gradients are not mysterious — they are flat regions.
• Exploding gradients are not chaotic — they are steep curvature.
• Learning rate schedules are not heuristics — they are step-size control mechanisms.
• Regularization is not magic — it reshapes the landscape.

This geometric view is far more powerful than memorizing optimizer names.

The Risk of Abstraction

Modern tooling is extraordinary. AutoML systems tune hyperparameters. Pre-trained foundation models dominate benchmarks. Fine-tuning APIs hide optimization entirely.

But abstraction creates fragility.

When a model:

• Refuses to converge
• Overfits immediately
• Becomes unstable during fine-tuning
• Fails to generalize in deployment

You need intuition — not documentation.

That intuition comes from fundamentals.

And fundamentals come from understanding optimization, not just calling it.

Research, Responsibility, and First Principles

In applied AI — particularly in healthcare and safety-critical systems — optimization errors are not academic.

They affect:

• Diagnostic performance
• Risk prediction
• False positive rates
• Real-world trust

If you treat gradient descent as a black box, you limit your ability to reason about model behavior under distribution shifts, data imbalance, or noisy supervision.

Understanding fundamentals makes you slower at first.

But more rigorous in the long run.

And rigor is what separates engineering from research.

Final Reflection

In the age of LLMs, fundamentals are not outdated.

They are leverage.

Anyone can fine-tune a model.
Fewer people can explain why it converges.
Even fewer can explain why it fails.

The next generation of AI researchers and engineers will not be defined by how well they use tools.

They will be defined by how deeply they understand the principles beneath them.

And beneath modern AI, the principle is still simple:

Make a prediction.
Measure error.
Follow the slope.
Repeat.

Emotion recognition has become one of the most promising frontiers of artificial intelligence — promising systems that don’t just parse language, but “feel” context, tone, expression, and intent all at once. Yet despite advances in deep learning and multimodal fusion, real-world emotion understanding remains elusive. In 2025, research shows that current multimodal emotion models are powerful yet brittle, and often fall short when deployed beyond carefully curated datasets.

In this article, I explore where today’s models fall short — and how the next generation of methods might finally bridge the gap between recognition and true understanding.

1. Multimodal Fusion Still Struggles with Contextual Ambiguity

Today’s state-of-the-art multimodal emotion models combine text, audio, and visual cues using deep learning. Yet these systems often fail to interpret context, resulting in misclassification when emotions depend on interaction history or nuanced situational factors.

Studies on dialogue-based emotion recognition demonstrate this limitation clearly. For example, in conversational settings using Transformer-based audio + text fusion, deep learning models outperform single modalities but still misinterpret emotions without broader context cues — showing how partial fusion leaves gaps in emotional understanding.

This reflects a deeper issue: because most training data — like IEMOCAP or EMOV — are lab-recorded and relatively static, models rarely encounter the fluid, ambiguous emotional expressions found in real life.

2. Static Labels Oversimplify Dynamic Emotions

Emotion is temporal by nature — it evolves over a conversation or interaction. However, most availability datasets and benchmarks train models to assign a single label to discrete snippets, failing to capture temporal emotional flux.

Advanced research recognizes this problem. Emerging frameworks such as Hierarchical Mixture-of-Experts (Hi-MoE)explicitly model continuous emotion streams and handle incomplete or asynchronous inputs, showcasing better resilience in dynamic environments. (arXiv)

This shift highlights how emotion dynamics matter — and how static labels flatten rich temporal cues that humans use naturally.

3. Missing & Noisy Modalities Still Break Systems

Real world sensing rarely guarantees high-quality text, audio, and video all at once. Environmental noise, occlusions, or sensor failure cause missing or degraded modalities. Most models assume full input availability and fail dramatically under missing signals.

Recent work analyzing robustness under modal absence shows that uncertain or missing data remains a core challenge in multimodal emotion AI, with standard fusion strategies unable to gracefully degrade.

This gap is not just theoretical — it’s why affective systems that work well in controlled labs often falter in actual use.

4. Datasets Bias Affect Outcomes

The quality and coverage of emotion data hugely impact model performance. A new enhanced valence–arousal multimodal dataset was released in 2025 to enrich emotional cues across continuous dimensions, aiming to reflect human experiential nuance better than coarse categorical labels. (Nature)

Still, even with richer annotation schemes, models struggle with cross-cultural expressiveness and context diversity — problems rooted in dataset biases and limited annotation frameworks.

5. Large Models Still Don’t Fully Grasp Emotion Semantics

While multimodal large language models (MLLMs) promise more flexible integration of modalities and broader reasoning capabilities, their emotion understanding is still early and inconsistent.

A 2025 benchmark of open-vocabulary emotion recognition shows that even the top MLLMs have a narrow advantage over traditional models, particularly when it comes to nuanced or non-categorical emotional states. (arXiv)

This suggests that while large models excel at general multimodal fusion, specialization for emotion reasoning remains an open challenge.

So, What’s Next?

Today’s research points toward several emerging directions that could overcome existing limitations:

A. Dynamic Emotion Modeling

Moving away from single-shot classification to continuous, time-aware affect representations enables systems to track emotional changes across interactions. Frameworks like Hi-MoE show that temporal continuity is achievable — and necessary. (arXiv)

B. Context + History Integration

Future systems should incorporate broader context — conversational history, situational cues, and even personal or cultural background — enabling deeper interpretive insight rather than surface recognition.

This parallels trends in dialogue systems research that emphasize contextual reasoning as core to human-like understanding.

C. Uncertainty-Aware Predictions

Humans rarely assign discrete emotional states; an utterance may be 30% frustration, 20% amusement, and 50% contemplation. New datasets and prediction frameworks are beginning to embrace emotion distributions instead of hard labels. (Nature)

D. Cross-Cultural & Multilingual Affect

Most emotion datasets and models remain heavily skewed toward Western, English datasets. Truly robust emotion AI will require frameworks that understand cultural variation in expression — a direction gaining traction in recent surveys. (MDPI)

E. Robust Fusion Under Real Constraints

Building systems that can handle missing, noisy, or asynchronous modalities — like the proposed hierarchical fusion mechanisms — is essential for real-world deployment. (arXiv)

Conclusion

Multimodal emotion analysis has come far — but it is still only scratching the surface of human affective complexity. In 2025, research highlights both progress and persistent limitations: static labels, context neglect, modality fragility, semantic gaps, and dataset bias.

The next generation of emotion models will need to treat emotion as fluid, contextual, uncertain, and deeply human — not just a category to be classified.

The future of affective AI isn’t merely predicting emotion — it’s understanding it.

Language is not just text ; it is structure, culture, and intent compressed into symbols. For humans, this compression is effortless. For machines, it is anything but.

At the heart of Natural Language Processing (NLP) lies a deceptively simple question: how does a model represent language? Not how it translates, classifies, or summarizes, but how it internalizes meaning in the first place. This question becomes especially interesting in multilingual settings, where languages differ not only in vocabulary, but in writing systems, syntax, and semantic density.

Consider English and Japanese. English relies on spaces, relatively simple morphology, and a fixed alphabet. Japanese weaves meaning across Kanji, Hiragana, and Katakana ; often without explicit word boundaries. And yet, modern NLP systems are expected to understand both, sometimes simultaneously, within a single model.

This article explores how multilingual NLP models learn representations across such linguistic boundaries; how Kanji becomes vectors, how meaning becomes geometry, and how gradients quietly align languages into a shared semantic space.

Why Representation Learning Matters

Raw text is meaningless to a machine. A sentence is not a sentence ; it is a sequence of characters or tokens that must be transformed into numbers before any learning can occur.

Representation learning is the process by which models convert discrete linguistic units into continuous, dense vectorsthat capture semantic and syntactic information. These vectors form the foundation for nearly every NLP task: sentiment analysis, translation, question answering, toxicity detection, and beyond.

Good representations have a powerful property: similar meanings map to nearby points in space. The sentence “I love this movie” should live closer to “This film was amazing” than to “I hated every minute of it”. In multilingual NLP, the challenge is even greater — the model must learn that 「この映画が好きです」 belongs in the same neighborhood as its English counterpart.

Language Is Not Universal: A Multilingual Reality

Languages encode meaning differently.

English is alphabetic, space-delimited, and relatively linear. Japanese, by contrast, is a hybrid system:

• Kanji carry dense semantic meaning
• Hiragana encodes grammar and inflection
• Katakana handles foreign words and emphasis

A single Kanji character can encapsulate what might require an entire English word or phrase. Word boundaries are often implicit, and meaning is heavily context-dependent. This makes even basic preprocessing (such as tokenization) non-trivial.

For multilingual NLP models, this diversity poses a fundamental question:
How can one representation space accommodate radically different linguistic structures?

From Tokens to Vectors: How Models “See” Language

Before models can learn meaning, text must be broken down into tokens. Early NLP systems relied on word-level tokenization, but this quickly broke down in multilingual settings ; especially for languages like Japanese.

Modern models instead rely on subword tokenization techniques such as Byte Pair Encoding (BPE) or WordPiece. These approaches strike a balance: they decompose rare or complex words into smaller units while preserving frequently occurring patterns.

For Japanese, this often means breaking text into sub-character or subword units that allow the model to:

• Reuse representations across words
• Handle unseen vocabulary
• Learn shared semantic components across scripts

Once tokenized, each unit is mapped to an embedding vector — a learned numerical representation. Early embeddings (like Word2Vec or GloVe) were static: each word had one vector, regardless of context. Modern transformer-based models generate contextual embeddings, meaning the representation of a token changes depending on its surrounding words.

The Kanji 「生」, for example, means different things depending on context — life, raw, birth. Contextual embeddings allow the model to reflect these nuances dynamically.

One Space, Many Languages: Multilingual Embeddings

Multilingual models such as mBERT and XLM-RoBERTa are trained on text from dozens (or hundreds) of languages simultaneously. Remarkably, without explicit alignment, these models often learn a shared semantic space.

In this space:

• English and Japanese sentences with similar meanings cluster together
• Language identity becomes a secondary feature
• Semantics dominate syntax

This phenomenon enables zero-shot cross-lingual transfer; a model trained on English sentiment data can often classify Japanese sentiment reasonably well, even without Japanese labels.

Meaning, not language, becomes the organizing principle.

From Kanji to Gradients: What the Model Actually Learns

So how does this alignment happen?

The answer lies in gradients.

During training, the model makes predictions and compares them against a loss function. The error signal propagates backward through the network, adjusting weights to reduce future loss. Over millions of examples, the model learns which patterns matter; and which don’t.

For multilingual models, this means:

• Learning shared semantic features across languages
• Mapping structurally different inputs to similar latent representations
• Abstracting away from surface-level form

The model is not “learning Japanese” or “learning English” in the human sense. It is learning relationships, between tokens, contexts, and outcomes. Kanji are not symbols; they are coordinates shaped by gradients.

Where This Breaks (and Why That Matters)

Despite their elegance, multilingual representations are not perfect.

• Low-resource languages are often underrepresented
• Cultural nuances may collapse into oversimplified embeddings
• Politeness, honorifics, and formality in Japanese are poorly captured
• Biases present in high-resource languages can dominate the space

When languages share a representation space, power imbalances in data distribution inevitably surface. This makes evaluation, interpretability, and ethical deployment critical concerns.

Why This Matters Beyond Research

Multilingual representation learning is not just an academic exercise.

It enables:

• Cross-lingual sentiment and toxicity detection
• Global healthcare NLP systems
• Sustainability research using multilingual datasets
• AI systems that are accessible beyond English-centric contexts

In a connected world, language should not be a barrier to technology or understanding.

Closing Thoughts

From Kanji to gradients, multilingual NLP is a story of abstraction. Language becomes geometry. Meaning becomes proximity. Learning becomes alignment.

Representation learning reminds us that intelligence (artificial or otherwise) is not about memorizing symbols, but about understanding relationships. And as models continue to improve, the hope is not that they replace language, but that they help bridge it.

When we talk about multilingual language models, we often describe them using abstract terms like “shared representation space” or “semantic alignment.” These phrases sound convincing, but they remain intangible until we try to see what they mean.

This article is a small attempt to make that abstraction visible.

By projecting Japanese and English sentence embeddings into two dimensions using PCA, we can explore how multilingual models organize meaning across languages, not through metrics or benchmarks, but through geometry.

The goal here is not evaluation or performance comparison. It is intuition-building.

Embeddings as Geometry

At their core, embeddings are geometric objects.

A sentence embedding is simply a point in a high-dimensional space, positioned such that semantically similar sentences lie closer together than unrelated ones. Multilingual models extend this idea by attempting to place semantically equivalent sentences from different languages near each other in the same space.

In theory, an English sentence and its Japanese translation should occupy neighboring regions, even though the surface forms differ completely.

What PCA allows us to do is collapse this high-dimensional structure into something we can inspect visually, while accepting that some distortion is inevitable.

Why Visualize Multilingual Embeddings?

Quantitative evaluation tells us whether a model works. Visualization helps us reason about how it works.

A two-dimensional projection cannot preserve all relationships, but it can reveal:

• Whether embeddings cluster by language or by meaning
• Whether translation pairs remain close after projection
• Whether the representation space appears shared or fragmented

These insights are especially useful when working across languages with different scripts, syntactic structures, and semantic conventions, such as English and Japanese.

The Experimental Setup

For this exploration, I used a small set of semantically aligned English and Japanese sentences sourced from an open, publicly available parallel corpus.

Each sentence was encoded using a multilingual sentence embedding model that maps multiple languages into a shared vector space. These embeddings were then projected into two dimensions using Principal Component Analysis.

The objective here is not scale. It is clarity.

Implementation Snapshot

Rather than walking through every preprocessing step, this section focuses only on the core logic required to generate the visualization.

The following snippet shows:

• Encoding parallel English and Japanese sentences
• Applying PCA to the combined embedding space
• Plotting both languages together while connecting translation pairs

from sentence_transformers import SentenceTransformer
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
import numpy as np

# Parallel English–Japanese sentence pairs (illustrative subset)
english_sentences = [
    "I like coffee",
    "The train is late",
    "This book is interesting",
    "It is raining today",
    "She is studying machine learning"
]

japanese_sentences = [
    "私はコーヒーが好きです",
    "電車が遅れています",
    "この本は面白いです",
    "今日は雨が降っています",
    "彼女は機械学習を勉強しています"
]

# Load multilingual sentence embedding model
model = SentenceTransformer("paraphrase-multilingual-MiniLM-L12-v2")

# Encode sentences
emb_en = model.encode(english_sentences)
emb_ja = model.encode(japanese_sentences)

# Combine embeddings for joint PCA
embeddings = np.vstack([emb_en, emb_ja])

# Reduce to 2D using PCA
pca = PCA(n_components=2)
emb_2d = pca.fit_transform(embeddings)

# Split projected points
en_2d = emb_2d[:len(english_sentences)]
ja_2d = emb_2d[len(english_sentences):]

# Plot
plt.figure(figsize=(7, 6))
plt.scatter(en_2d[:, 0], en_2d[:, 1], label="English")
plt.scatter(ja_2d[:, 0], ja_2d[:, 1], label="Japanese")

# Connect translation pairs
for i in range(len(english_sentences)):
    plt.plot(
        [en_2d[i, 0], ja_2d[i, 0]],
        [en_2d[i, 1], ja_2d[i, 1]],
        linewidth=0.8
    )

plt.title("PCA Projection of English and Japanese Sentence Embeddings")
plt.legend()
plt.show()

For clarity, the visualization shown here uses a small illustrative subset of sentence pairs representative of such corpora.

Observations from the PCA Projection

The PCA projection of English and Japanese sentence embeddings is shown below. Each point corresponds to a sentence, and each line connects a translation pair.

Several observations can be made.

First, the embeddings do not exhibit a clear separation by language. This suggests that the dominant axes of variance are not primarily language-specific.

Second, many translation pairs remain spatially close even after projection, indicating that semantic alignment persists under dimensionality reduction.

Third, some translation pairs diverge noticeably. This is expected, as PCA preserves global variance rather than pairwise relationships, and fine-grained semantic distinctions may be lost in low-dimensional projections.

What this does and does not mean

It is tempting to overinterpret visualizations like this, so it is worth being explicit.

This plot does not prove that Japanese and English are perfectly aligned in embedding space. It does not measure translation quality. It does not tell us which language is easier or harder to represent.

What it does suggest is more modest and more useful. Multilingual embedding models appear to prioritize semantic relationships over surface linguistic features such as script or word order. At least in this projection, meaning seems to matter more than language identity.

At the same time, the distortions introduced by dimensionality reduction are clearly visible. Some translation pairs drift apart not because the model failed, but because PCA flattened a complex space into two dimensions.

Visualization builds intuition, not evidence.

Why this perspective matters

Thinking about embeddings geometrically changes how we reason about multilingual AI. Instead of asking whether a model “understands” different languages, we ask how it organizes meaning numerically.

This shift removes some of the mystery around multilingual models. What looks like linguistic intelligence is often the result of careful mathematical alignment. Vectors, distances, and projections do the heavy lifting.

Seeing this makes the system feel less magical, but also more honest.

A closing thought

PCA plots like this are not answers. They are lenses.

They remind us that language models operate in spaces defined by mathematics, not meaning. When different languages appear to coexist peacefully in those spaces, it is not because the model understands culture or grammar. It is because geometry allows similar patterns to align.

Understanding that does not diminish multilingual AI. It grounds it.

And sometimes, grounding is exactly what clarity requires.

Not because it became less impressive, but because I began to see what was happening beneath the surface. The more time I spent around AI systems, the clearer it became that they were not thinking, reasoning, or understanding in any human sense.

They were calculating.

That realization changed how I understood AI entirely. Math is not just a tool used in artificial intelligence. It is the language AI operates in.

When data stops being information

Humans experience data through meaning.
A sentence carries emotion.
An image tells a story.
A sound implies urgency or calm.

For an AI system, none of that exists.

Before a model can work with anything, it must first be translated into numbers. Text, images, audio, and signals all become numerical representations. This is not a technical detail. It is a requirement.

Once data becomes numbers, it becomes geometry.

Words turn into vectors.
Images become matrices.
Patterns become positions in high dimensional space.

In this space, meaning is defined by distance. Similar things are closer together. Different things are farther apart. AI does not interpret meaning. It measures relationships.

Without mathematics, AI has no way to represent the world at all.

Learning is optimization, not understanding

When we say an AI model is learning, it is easy to imagine insight or comprehension taking place. In reality, learning is far more mechanical.

A model makes a prediction.
That prediction is compared to reality.
The difference becomes a number called an error.

From there, math takes over.

The system adjusts its internal parameters slightly to reduce that error. Then it repeats the process. Again and again.

This loop is driven by optimization and calculus. Gradients point toward improvement. Step sizes control stability. Convergence is a mathematical property, not a sign of intelligence.

AI does not know when it is right. It only knows when it is less wrong.

Intelligence, but probabilistic

AI systems often appear confident. They return clear answers, labels, or predictions. But beneath that confidence lies uncertainty.

Almost everything an AI system produces is probabilistic.

It does not say, “This is correct.”
It says, “This is the most likely option given the data.”

Every output exists near a decision boundary. Confidence scores, likelihoods, and distributions are not optional features. They are the core of how AI reasons.

Humans rely on context, experience, and intuition. AI relies on numbers, distances, and probabilities. Math forces AI to be honest about uncertainty, even when interfaces hide it.

Why constraints matter more than cleverness

It is tempting to believe that more complex models automatically lead to better intelligence. Math quickly challenges that assumption.

Every AI system operates under constraints.
* Limited data.
* Limited computation.
* Limited tolerance for error.

These constraints are mathematical realities. Trade offs between accuracy and generalization, simplicity and performance, efficiency and capability are governed by theory, not preference.

Sometimes a simpler model performs better in practice. Not because it is smarter, but because it aligns better with the problem’s constraints. Math does not just enable AI. It disciplines it.

The more time I spend with AI, the less I think of it as artificial intelligence and the more I think of it as applied mathematics at scale.

That perspective does not diminish what AI can do. It grounds it.

If you want to truly understand AI, not just use it or prompt it, you eventually have to learn its language.

That language is math.

I started studying Japanese nearly four years ago out of personal interest, for reasons entirely unrelated to machine learning. What I did not expect was that, years later, it would fundamentally alter how I think about Natural Language Processing. In particular, it exposed how many so-called standard NLP abstractions are not universal properties of language, but artifacts of English.

Most NLP systems follow a familiar pipeline. Text is tokenized, mapped to embeddings, and passed through a model optimized for a downstream task. This workflow feels language-agnostic. Learning Japanese made it clear that many of its assumptions only hold because English conveniently satisfies them.

English-Centric Assumptions in NLP

A large fraction of NLP intuition relies on implicit assumptions:

• Word boundaries are clearly defined
• Tokens approximate semantic units
• Syntax explicitly encodes relationships
• Linear order carries most of the meaning

These assumptions hold well for English and other whitespace-delimited languages. Even subword tokenization preserves the intuition that language can be segmented into relatively stable units before modeling begins.

Japanese violates all of these assumptions simultaneously.

Tokenization as a Modeling Decision

Japanese text has no spaces between words, and a single sentence can often be segmented in multiple valid ways depending on interpretation. There is no canonical “correct” tokenization independent of context.

As a result, tokenization ceases to be a neutral preprocessing step. It becomes an explicit modeling choice that constrains the hypothesis space before learning even begins.

This reframed how I think about tokenizers. Instead of viewing tokens as atomic carriers of meaning, I began to see them as inductive biases; choices that shape which interpretations a model can efficiently represent.

Context, Omission, and Implicit Meaning

Japanese frequently omits subjects, objects, and other syntactic elements when they are pragmatically inferable. Meaning is reconstructed using discourse context rather than explicit surface structure.

Many NLP datasets and benchmarks implicitly assume explicit predicate–argument structure. Japanese reveals how fragile those assumptions are. Much of human language understanding operates on inferred, not stated, information.

From a modeling perspective, this highlights a gap between syntactic completeness and semantic sufficiency — a gap that accuracy-based metrics often fail to capture.

Ambiguity Is Not an Error

Another shift was how I think about ambiguity. In Japanese, ambiguity is often intentional and only resolvable through context. Multiple interpretations can be valid until disambiguated by downstream information.

This has direct implications for evaluation. When a model produces an output that differs from a reference, is it hallucinating; or is it selecting a different, but plausible, interpretation?

Japanese makes it clear that many “errors” in NLP are actually mismatches between evaluation assumptions and linguistic reality.

Implications for Modern NLP

These insights matter directly for:

• Multilingual and cross-lingual models
• Low-resource language transfer
• Tokenizer design
• Prompt sensitivity in large language models
• Evaluation of generative systems

English-centric intuitions silently influence architecture choices and benchmarks. Languages like Japanese surface these blind spots early by violating multiple assumptions at once.

Conclusion

Studying Japanese didn’t just give me exposure to a new language. It forced me to interrogate what NLP systems assume about language in the first place.

I now think of language models less as text processors and more as systems that negotiate meaning under uncertainty. If we want models that generalize across languages (rather than perform well on English-shaped benchmarks ) we need to take those uncertainties seriously.