Data Modeling evolution
Five eras of data modeling
People in different eras developed techniques for specific problems. Once the technique was built they stick to that paradigm and thought it matters the most. Each one of us probably grew up in one of these eras. All eras have produced vital work, and all have massive blind spots.
The Relational Era
1970s
- Normal form
- Referential integrity
- Eliminating redundancy
This is the thing that we learn first in CS or database theory.
Blind Spot: A schema requiring seven joins for a basic query is pure on paper and painful in production.
The Analytics Era
1990s
- Inmon’s enterprise data warehouse
- Kimball’s dimensional modeling
- Linstedt’s Data Vault
Blind Spot: No support for real-time event streams, graph traversals, or feature tables
The Application Era
2000–2010s
- Speed and flexibility
- NoSQL, MongoDB, Redis, Cassandra, Kafka.
- Data model around app access patterns. Fetches everything in a single read.
Blind Spot: Schemas tuned for one app’s reads create chaos when analytics needs millions of records or ML needs clean feature extraction.
The ML/AI Era
2010–2020s
- Feature tables
- Embeddings
- Vector spaces
It organizes data for training and inference.
Blind Spots: Data leakage between training and test set. Lack of Entity Resolution results in duplicated customer records used in training.
The Knowledge Era
2020-now
- ontologies
- taxonomies
- knowledge graphs
What is a “Customer”? Is an “Order” an entity, a state, or an event?
Core Problem: Isolation
We rarely look beyond our niche.
- Data engineers who’ve never designed a normalized table.
- App devs who’ve never heard of slowly changing dimensions.
- ML engineers who’ve never modeled a core business domain.
- Ontologists who’ve never shipped a production pipeline.
Specialization worked when systems were isolated. Today, an e-commerce stack runs Postgres for orders, Kafka for events, star schemas in the warehouse, embeddings in vector stores, and a semantic layer defining “customer” and “revenue.” It exists simultaneously. It all gets modeled: intentionally or accidentally.
Meeting points
Operations meet analytics (1990s–2000s).
Data modeling meant relational design for applications. Then business questions started hitting production OLTP databases. We had to map normalized transactional sources into analytical targets.
Big data and lakes (2000s–2010s)
Hadoop, NoSQL, JSON, and event streams brought schema-on-read.
Dump it in the lake; we’ll figure it out later.
Unmodeled lakes turned into data swamps. Successful engineers had to connect app documents, event streams, and warehouse models.
ML and AI (2010s–present)
A customer is now an OLTP record, a warehouse dimension, a feature vector, and a graph node. These representations must agree. An AI agent facing ambiguous columns, missing business rules, or broken grain generates confident garbage at machine speed.
Eras, Shape, and Modeller
Structured data spans two Eras. Thats why the friction was between Relational and Analytics practitioners for decades.
Get Amit Singh Rathore’s stories in your inbox
Join Medium for free to get updates from this writer.
Take a single real-world transaction: I buy a red kettlebell to support the University of Utah (go Utes!). Order 808.
- Transactional: Normalized rows in Orders and OrderLines ensuring inventory integrity.
- Event: Clickstream events. Searched “kettlebell,” added to cart, checked out. Streamed through Kafka.
- Analytical: A fact table record joined to a surrogate CustomerKey to track dimension changes over time.
- Unstructured: A product review with a chipped handle image evaluated by sentiment analysis.
- ML/AI: Feature vectors for recommendations placing me near strength gear and far from yoga mats.
- Knowledge: An explicit business rule requiring a “Customer” to have at least one completed purchase, preventing AI agents from counting newsletter subscribers as active buyers.
One order, six representations. The CustomerID in OLTP must resolve to the CustomerKey in the warehouse, the user ID in Kafka, and the feature vector. When those keys break, your architecture falls apart.
Cross disciplinary literacy: Beome T-shapes
We don’t need master every discipline. We need cross-disciplinary literacy. Expert in one and competent in others.
A relational modeler who understands dimensional modeling builds source schemas that load cleanly into data warehouses. A data engineer who understands feature stores builds pipelines ML teams can actually use. A practitioner who understands semantics documents business logic so schemas explain themselves to both humans and LLMs.
Stop building models that solve problems for your team while creating technical debt for others.
