DynamoDB Internals

I have been diving deeper into DynamoDB lately and, honestly, I found it pretty difficult to understand at first. Not because the APIs are complicated, but because of the way it forces you to think about data modeling. I come from a traditional SQL background, where we usually design databases from the bottom up. First we define entities and relationships, then normalize everything till 3NF, and later add denormalization wherever joins become expensive. After that, we build the APIs and application logic around the schema. That flow feels intuitive because relational databases are designed for it.

With NoSQL databases, especially Amazon DynamoDB, things feel almost reversed. You cannot really design the database properly unless you already know your access patterns. You need to think about the application layer, controller logic, and query patterns before even deciding how the data should be stored. Coming from SQL, that felt very unnatural to me.

You can read the following thread where I do deeper into this

Then there are DynamoDB-specific concepts like RCUs, WCUs, partition keys, hot partitions, and optional strongly consistent reads. When you first encounter these, they honestly feel weird. Why should reads and writes have “capacity units”? Why should traffic distribution affect schema design? Why is consistency configurable per request? A lot of DynamoDB advice online tells you what to do, but not why those constraints exist in the first place.

That curiosity is what pushed me to dive deeper into DynamoDB internals. I wanted to understand the engineering decisions behind these design choices and how the internal architecture of DynamoDB shapes the way applications are built around it. This blog is my attempt to connect those dots.

If this is your first time reading my blog, welcome! I usually write about microservices, databases, distributed systems, and other engineering topics that I find interesting. If you enjoy this article, feel free to follow along and connect with me on other platforms as well :)

Introduction

Amazon DynamoDB is a fully managed NoSQL database service offered by Amazon Web Services (AWS). It is designed to deliver fast, predictable, and scalable performance for applications that require low-latency data access at any scale.

AWS provides very high availability guarantees for DynamoDB through its Service Level Agreement (SLA).

1. 99.99% availability for regular tables
2. 99.999% availability for global tables

It has both — Strong and Eventual Consistency
One of DynamoDB’s standout features is its ability to deliver single-digit millisecond latency at virtually any scale. This means read and write operations typically complete within just a few milliseconds, making DynamoDB suitable for real-time applications such as gaming, IoT, recommendation systems, and high-traffic web applications.

Data Model and Distribution

Let’s first take a look at DynamoDB’s structure and understand why some of its seemingly “strict” constraints are actually enforced by the underlying architecture.

A DynamoDB table is essentially a collection of items, where each item is itself a collection of attributes. Every item is uniquely identified using a primary key, and this key schema must be defined when the table is created.

At first glance, this can feel restrictive, especially if you come from the SQL world where indexing strategies can evolve later. But DynamoDB’s design choices make much more sense once you look under the hood.

Table divided into Partitions

Internally, a single DynamoDB table is split into multiple partitions. Each partition stores a disjoint subset of the table’s data and is responsible for a contiguous range of keys. The partition key plays a critical role here because it determines where an item will be physically stored.

This partitioning serves two major purposes:

1. Handling higher throughput as request traffic increases
2. Storing larger amounts of data as the table grows

You can think of it like this: if a table contains 100 keys and is divided into 5 partitions, then each partition may roughly manage around 20 keys. In reality, the distribution is hash-based rather than sequential, but conceptually the workload is spread across multiple physical storage units.

Each partition is also replicated across multiple Availability Zones for fault tolerance. DynamoDB maintains a replication factor of 3, meaning every partition typically exists on three different storage nodes distributed across separate Availability Zones. Together, these replicas form what DynamoDB refers to as a replication group.

Partitions P1 P2 P3 Split Across Multiple AZs with Replication Factor = 3

This is what gives DynamoDB its cross-AZ fault tolerance. A write is acknowledged once 2 out of 3 replicas (a quorum) persist it to their WALs — so you can lose one replica and still make progress.

Note: For global tables, each AWS Region maintains its own 3-replica replication group, and changes are asynchronously replicated across Regions. So if you have a global table in 3 regions, you effectively have 9 replicas total — though cross-region replication is eventual, not synchronous.

We will dive deeper into replication later, but for now, the important thing to understand is this:

The partition key is not just a logical identifier. It is the mechanism DynamoDB uses to physically distribute and scale data across machines.

Internally, the partition key is hashed to determine where the item should live.

Partition Key
      ↓
Internal Hash Function
      ↓
Partition Identifier
      ↓
Physical Storage Partition / Server

If you are familiar with distributed systems, you might assume DynamoDB uses ring-based consistent hashing like the original Dynamo paper. And historically, you would have been correct.

The original Dynamo system used a consistent hashing ring with virtual nodes (vnodes):

• All nodes were placed on a circular ring
• Each key was hashed onto the ring
• Data was stored on the first node clockwise from the hash position
• Nodes owned multiple virtual nodes for better load balancing
• When nodes were added or removed, only a small portion of data needed redistribution

This design was elegant because scaling events required minimal data movement.

However, modern DynamoDB moved away from the peer-to-peer ring architecture.

When Amazon built DynamoDB as a fully managed service, they adopted a more centrally orchestrated partitioning model. While the exact hashing implementation is not publicly documented, we do know a few important details from AWS papers and talks:

• DynamoDB still hashes the partition key to determine placement
• But partitions are managed as explicit units rather than positions on a decentralized ring
• An internal service called AutoAdmin manages partition splitting, movement, and placement
• Metadata about partition ownership is stored in MemDS, an in-memory metadata service
• Request routers use this metadata to route requests to the correct storage partitions

Side Note: We will get into these terms later on. Don’t worry :)

So instead of nodes self-organizing like a distributed ring, DynamoDB operates more like a centrally managed distributed storage system where AWS controls placement and scaling decisions.

This architecture also explains why DynamoDB requires the primary key schema to be defined upfront. Since request routing, partition placement, replication, and scaling all fundamentally depend on the partition key, the database needs to know exactly how data will be identified before any data is stored.

The Replication Group & Reads/Writes

Let’s go back to the replication example for a moment.

We know that DynamoDB has replication factor of 3. Assume that Partition P1 is replicated across 3 storage nodes (say DataNode A, B, C) — one per AZ.

Together they form a Replication Group.

Within this replication group, one replica acts as the leader, while the remaining replicas act as followers. The entire coordination mechanism is governed using Multi-Paxos, which DynamoDB uses for consensus and leader election.

Any Replica can trigger the leader election, and the elected leader maintains leadership via a lease mechanism (it periodically renews; if it fails to renew, a new election is triggered).

Note: This detail becomes very important later when we discuss strongly consistent reads.

The leader has the following responsibilities
1. To serve all writes
2. To serve strong consistent reads

Followers can serve:
1. Eventually consistent reads only

This architecture is one of the reasons DynamoDB can achieve extremely high read scalability. Since eventually consistent reads can be served by any replica in the replication group, read traffic can be distributed across multiple nodes instead of bottlenecking on a single machine.

However, there is a tradeoff :)

Because follower replicas may lag slightly behind the leader, they might temporarily return stale data. That is exactly why DynamoDB differentiates between:

• Eventually consistent reads
• Strongly consistent reads

Strongly consistent reads must go through the leader because only the leader is guaranteed to have the latest committed state. Followers may still be catching up through replication.

Here’s a snippet of Strong Consistent Read in Golang

import (
    "context"
    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)

func getItemStronglyConsistent(client *dynamodb.Client, userID string) (*dynamodb.GetItemOutput, error) {
    result, err := client.GetItem(context.TODO(), &dynamodb.GetItemInput{
        TableName: aws.String("Users"),
        Key: map[string]types.AttributeValue{
            "UserID": &types.AttributeValueMemberS{Value: userID},
        },
        ConsistentRead: aws.Bool(true), // Forces request to partition leader
    })
    if err != nil {
        return nil, err
    }
    return result, nil
}

So when DynamoDB asks whether you want a “Consistent Read,” it is not just an API-level option. It directly affects which replica serves your request and how the distributed consensus system behaves underneath.

What ConsistentRead Forces Internally

Simplified View (It about to get a lot more complicated trust me :))

When a request router sees ConsistentRead: true, it cannot simply forward the request to any replica. Instead, it must first identify the current leader of that partition’s replication group and route the request specifically to that node.

Cost Implications
Strongly consistent reads cost 2x more RCUs than eventually consistent reads.
For example:
Reading a 4 KB item with eventual consistency costs 1 RCU
Reading the same item with strong consistency costs 2 RCUs

Why? Because eventually consistent reads can be distributed across followers, while strongly consistent reads must hit the leader directly. You are essentially paying for stronger coordination guarantees.

Latency Implications
Strongly consistent reads also tend to have slightly higher and less predictable latency.
With eventually consistent reads, DynamoDB has flexibility. The request router can send your request to the nearest replica or the least loaded follower. But with strong consistency enabled, the router loses that flexibility because only the leader is allowed to serve the request.

That means:

• Every consistent read targets one specific node
• Your read now competes with write traffic happening on the same leader
• Heavy write activity can directly affect consistent read latency

So the latency difference is not just an implementation detail. It is a direct consequence of the replication architecture.

Availability Implications
Suppose the leader crashes or becomes unhealthy. The replication group now needs to elect a new leader using Multi-Paxos. During this election window:

• The old leader’s lease must expire
• A new leader must be elected
• The new leader must establish authority before serving traffic

This process usually completes within a few seconds, but during that window, strongly consistent reads may fail or temporarily block because there is no confirmed leader available yet.

Eventually consistent reads, however, can still succeed because followers are still alive and capable of serving reads.

So technically speaking, enabling ConsistentRead: true slightly reduces practical availability in exchange for stronger correctness guarantees.

Throughput Implications
There is another scaling implication that becomes important at large traffic volumes. All strongly consistent reads for a partition are funneled through a single node: the leader.

That leader already handles:

• All writes
• Replication coordination
• Consensus responsibilities
• Lease renewals

At high scale, the leader itself can become a hotspot even if follower replicas are mostly idle. This is one of the reasons DynamoDB encourages eventual consistency wherever possible. The architecture is optimized around distributing read traffic across replicas.

Overusing strongly consistent reads can unintentionally create bottlenecks on otherwise healthy partitions.

Global Secondary Indexes and Local Secondary Indexes

Global Secondary Indexes are probably one of the most important features in Amazon DynamoDB, but they are also one of the most misunderstood.

A GSI is a separately partitioned, separately replicated index that DynamoDB maintains internally.

You can think of them as “tables” that you don’t manage. DynamoDB maintains it for you automatically.
Each GSI has:

• Its own partition key
• An optional sort key
• Its own partitions
• Its own replication groups
• Its own throughput allocation
• Its own storage nodes

When a write happens on the base table, DynamoDB asynchronously propagates the relevant attributes to the GSI’s storage partitions. That “asynchronously” part is extremely important because it explains almost every design tradeoff and restriction around GSIs.

Typically the replication lag is tiny, often milliseconds, but there is no strict guarantee.

func getOrdersByStatus(client *dynamodb.Client, status string) (*dynamodb.QueryOutput, error) {
    result, err := client.Query(context.TODO(), &dynamodb.QueryInput{
        TableName: aws.String("Orders"),
        IndexName: aws.String("StatusIndex"), // GSI name

        KeyConditionExpression: aws.String("#s = :statusVal"),

        ExpressionAttributeNames: map[string]string{
            "#s": "Status", // GSI partition key
        },

        ExpressionAttributeValues: map[string]types.AttributeValue{
            ":statusVal": &types.AttributeValueMemberS{
                Value: status,
            },
        },

        // ConsistentRead: aws.Bool(true)
        // Not allowed on GSIs. DynamoDB throws ValidationException.
    })

    if err != nil {
        return nil, err
    }

    return result, nil
}

What Actually Happens During a GSI Query?

Internally, the flow looks something like this:

1. The request arrives at the request router
2. The router notices IndexName: "StatusIndex"
3. The router consults MemDS to determine which GSI partition owns the requested key range (Don’t worry about MemDS, we’ll get to what it is)
4. The request is routed to the appropriate GSI storage node
5. The GSI replica serves the query

The important detail is this:

The request never touches the base table partition directly.

You are querying an entirely separate distributed storage structure maintained asynchronously from the base table.
That means the GSI may temporarily lag behind recent writes.
This is why ConsistentRead: true on a GSI is not merely “more expensive.” DynamoDB outright rejects it with a ValidationException.
There is no “strong vs eventual” consistency choice for GSIs. There is only eventual consistency.

Why LSIs Support Strongly Consistent Reads

Local Secondary Indexes (LSIs) work very differently.
Unlike GSIs, LSIs share the same partition key as the base table. That single design decision changes everything internally.
Because LSIs share the same partition key:

• They live on the same physical storage partition as the base table
• They belong to the same replication group
• They share the same leader replica

Most importantly:

LSI updates happen synchronously as part of the original write operation itself.

When the leader processes a write:

• Base table data is updated
• LSI entries are updated
• Both changes are committed atomically together

Conceptually, they are part of the same write-ahead log (WAL) entry.
So when a strongly consistent read goes to the leader, the LSI is already guaranteed to reflect the latest committed state.

There is no async propagation delay.
There is no replication lag between the base table and the LSI.
That is why LSIs support strong consistency while GSIs do not.

BUT LSIs Have So Many Restrictions (Tradeoffs again :) )

At first glance, LSIs seem better:
1. Strong consistency
2. Atomic updates
3. Same-partition locality
But these advantages come at a very serious costs.

1. LSIs Must Be Defined at Table Creation
Unlike GSIs, LSIs cannot be added later.
Why? Because the LSI is physically co-located with the base table partition itself. DynamoDB needs to allocate storage structures for the LSI when the partition is initially created.

Once partitions already exist and contain data, DynamoDB cannot safely reorganize the storage layout retroactively to inject a new co-partitioned index. So this restriction is not an arbitrary product limitation.
It comes directly from the storage architecture.

2. The 10 GB Item Collection Limit
This is the most important LSI restriction. Because the LSI lives inside the same partition as the base table:

• Base table items
• LSI entries
• All items sharing the same partition key

must physically fit together within one partition.
And DynamoDB partitions have storage limits. Without LSIs, DynamoDB can split partitions freely as data grows.
But with LSIs, partition splitting becomes much harder because the LSI data must remain co-located with the base table data.

That is why DynamoDB imposes the famous:

10 GB item collection limit per partition key value when LSIs exist.

If that limit is exceeded, writes fail with: ItemCollectionSizeLimitExceededException

This is one of those DynamoDB constraints that sounds random until you understand the physical storage model underneath.

3. Maximum of 5 LSIs per Table
Again, this comes from co-location costs.
Every additional LSI:

• Increases partition storage overhead
• Increases synchronous write amplification
• Makes partition management harder
• Increases replication complexity

So DynamoDB keeps the number intentionally small.

4. LSIs Share Throughput with the Base Table
Unlike GSIs, LSIs do not have independent throughput allocation.
Why? Because they are not separate storage systems.
An LSI read or write ultimately hits the same partition leader and the same replication group as the base table itself.

So capacity is shared. GSIs, on the other hand, are separate distributed structures with separate scaling and throughput management.

The Tradeoff Between GSIs and LSIs

Once you understand the internals, the tradeoff becomes much clearer.
LSIs give you:

• Strong consistency
• Atomic updates
• Same-partition locality

But they also trap you inside the partition. Everything for a partition key value must remain physically co-located:

• Base table rows
• LSI data
• Replication state

That makes LSIs risky for workloads with large or fast-growing partition key collections. The 10 GB limit sounds generous until you hit a hot partition.

The Architecture of DynamoDB

Why we call DynamoDB a “Managed Service”

Here’s an overview of the architecture (Told you it gets complicated :) )

The Architecture of DynamoDB

1. The Front Door: Request Router

When a client makes an API call to DynamoDB, the request first hits the Request Router.
The router itself does not store any data. Its responsibility is to:

• Validate the request
• Authenticate and authorize the client
• Determine which physical partition owns the requested data
• Route the request to the correct storage nodes

At DynamoDB scale, even request routing becomes a distributed systems problem. Millions of requests per second cannot afford centralized bottlenecks or slow metadata lookups.

That is why DynamoDB’s request layer is designed to be lightweight, horizontally scalable, and aggressively cached.

2. The Control Plane: Validation and Routing

Before the router forwards traffic to the storage layer, it coordinates with several critical control-plane systems.

IAM Authorization

DynamoDB integrates deeply with AWS IAM, so every request must be checked to ensure the caller has permission to perform the operation.
But there is an important distributed systems design decision here.
If the request router had to synchronously contact the IAM service for every single request, then a temporary IAM slowdown could directly impact DynamoDB availability.

To avoid this, DynamoDB uses what AWS calls a statically stable architecture.

Request routers cache IAM authorization results locally so they can continue operating even if downstream services experience temporary failures. This allows DynamoDB to remain highly available even when parts of the broader AWS control plane are degraded.

This is a recurring theme in DynamoDB’s design:

Dependencies should never become availability bottlenecks.

Global Admission Control (GAC)

Next comes capacity management.
Early Dynamo-style systems suffered badly from hot partition problems because throughput enforcement happened at individual partitions.

DynamoDB evolved beyond that using a system called Global Admission Control (GAC).
Instead of rigidly rate-limiting each partition independently, GAC acts more like a centralized token bucket system for the table as a whole.

This allows:

• Underutilized partitions to “lend” unused capacity
• Hot partitions to temporarily burst above their baseline allocation
• Better overall throughput utilization across the table

This is one of the reasons modern DynamoDB handles uneven traffic patterns much better than many people assume.
The partition still has physical limits, but capacity management is now globally coordinated rather than statically partitioned.

MemDS &The Metadata Service

Now the router needs to determine where the data physically lives.
At DynamoDB scale, this is surprisingly difficult.
A naïve design would require every router to maintain huge routing tables mapping:

• Partition keys
• Partition ranges
• Replica locations
• Leaders
• Storage nodes

But large routing tables create cache invalidation problems and cache stampedes whenever partitions split or move.

DynamoDB solves this using MemDS, a highly optimized distributed in-memory metadata service.

Get Tirthraj Mahajan’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

MemDS stores:

• Partition ownership
• Replica membership
• Leader information
• Partition movement metadata

Routers query MemDS to determine the exact storage location for a request.

Because MemDS itself is horizontally distributed and heavily optimized for low-latency lookups, routing decisions remain extremely fast even as the storage fleet grows massively.

3. The Data Plane: Storage Nodes

Once the router:

• Validates permissions
• Checks capacity availability
• Resolves partition ownership

it forwards the request to the storage layer.
This is the actual heart of DynamoDB.

Each partition belongs to a replication group containing multiple replicas spread across Availability Zones.
Within that replication group:

• One replica acts as the leader
• Other replicas act as followers

The leader is responsible for:

• Processing writes
• Coordinating replication
• Serving strongly consistent reads

Storage Nodes and Log Nodes in DynamoDB

Write-Ahead Log (WAL)

When a write reaches the leader, the first thing DynamoDB does is append the operation to the Write-Ahead Log (WAL).

This is critical for durability.
The WAL ensures the operation is safely recorded before it is applied to the actual storage structures.

The leader then replicates the WAL entry to follower replicas using Multi-Paxos coordination.
Once a quorum acknowledges the write:

• The write is considered durable
• The operation is committed
• The local storage structures are updated

This is how DynamoDB achieves fault tolerance while still maintaining extremely low latency.

B-Trees and Storage Structures

After the write is committed through consensus, the data is applied to the underlying storage structures, typically B-tree-based key-value storage systems.

At a high level:

• WAL provides durability
• Paxos provides consensus
• B-trees provide efficient local storage and retrieval

Together they form the foundation of DynamoDB’s storage engine.

4. Extensions: Transactions, Backups, and Streams

One of the most impressive aspects of DynamoDB’s architecture is that advanced database features are implemented as separate subsystems layered around the core data plane rather than slowing the hot write/read path itself.

Transactions

DynamoDB originally avoided multi-item transactions entirely because distributed transactions are expensive.
Eventually AWS introduced ACID transactions, but notice how they are implemented:

• As a separate coordination layer
• On top of the existing storage engine

The Transaction Manager and Lock Manager coordinate:

• Two-Phase Commit (2PC)
• Item-level locking
• Conflict detection
• Atomic commit guarantees

This is why DynamoDB transactions:

• Cost 2x capacity
• Increase latency
• Reduce throughput

Distributed coordination is fundamentally expensive.

DynamoDB exposes transactions, but it clearly optimizes the system around the non-transactional fast path.

If you are intrested in 2PC — You can check this video out

Backup and Point-In-Time Recovery (PITR)

Backups are handled through asynchronous WAL archiving.
As WAL entries are generated:

• They are continuously streamed to Amazon S3
• Recovery snapshots are constructed from the WAL stream
• Point-In-Time Recovery (PITR) becomes possible

Because backups are asynchronous:

• Live storage nodes avoid backup overhead
• Production traffic remains unaffected

This is a classic distributed systems pattern:

Move expensive work off the critical path.

DynamoDB Streams

DynamoDB Streams are another elegant subsystem. Whenever changes are committed:

• The WAL entries are also pushed into an asynchronous change stream
• Consumers can subscribe to those changes
• AWS Lambda or downstream services can react in real time

This effectively turns DynamoDB into an event source for event-driven architectures.

The important architectural point is:

Streams piggyback on the WAL.

DynamoDB does not duplicate work unnecessarily. The WAL already contains the ordered mutation log, so Streams simply expose it externally.

5. The Autoadmin Service: DynamoDB’s Central Nervous System

The final piece of the puzzle is probably the most important operationally. Internally, DynamoDB runs an enormous orchestration system called Autoadmin.
This service continuously monitors:

• Storage node health
• Replica health
• Partition heat
• Capacity utilization
• Hardware failures
• Fleet balancing
• Table growth

Hardware Failures and Replica Recovery

In self-hosted distributed databases:

• Disks fail
• Nodes die
• Engineers get paged at 3 AM

Someone has to:

• Replace hardware
• Restore replicas
• Rebalance shards
• Repair cluster topology

DynamoDB automates all of this.

If a storage node becomes unhealthy:

• Autoadmin detects it
• New replicas are provisioned automatically
• Data is replicated to healthy nodes
• Traffic is rerouted transparently

Most users never even notice.

Hot Partition Management

Autoadmin also continuously monitors partition heat.

Suppose:

• Multiple tenants become hot simultaneously
• One storage node starts experiencing heavy contention
• Latency begins increasing

The storage nodes report candidate partitions for movement.
Autoadmin then:

• Identifies healthier storage nodes
• Migrates partitions
• Rebalances traffic
• Redistributes load across the fleet

This is one of the reasons DynamoDB can maintain single-digit millisecond latencies even during massive traffic spikes.

The operational complexity is hidden behind the service boundary.

Storage Admission Control and Capacity Units

At its core, Amazon DynamoDB still runs on physical machines with finite CPU, memory, disk bandwidth, and network throughput.

No matter how “serverless” the API feels, DynamoDB ultimately needs a mechanism to prevent individual storage nodes from getting overwhelmed by traffic.

That mechanism is called Storage Admission Control.

Storage Admission Control

Instead of exposing hardware details directly, DynamoDB abstracts resource consumption into two logical units:

• Read Capacity Units (RCUs)
• Write Capacity Units (WCUs)

Read Capacity Units (RCUs)

An RCU defines how many reads per second DynamoDB can serve for a given item size and consistency level.

1 RCU:

RCUs to RPS

Notice something interesting here:

• Eventually consistent reads are cheaper
• Transactional reads are more expensive

And by now, we actually understand why.

Eventually consistent reads can hit follower replicas, distributing load across the replication group. Strongly consistent reads must go to the leader. Transactional reads involve even more coordination overhead.

The pricing model directly mirrors the internal distributed systems cost.

Write Capacity Units (WCUs)

Similarly, WCUs define write throughput.
1 WCU:

WCUs to RPS

Again, transactional writes cost more because they involve:

• Two-phase commit
• Lock coordination
• Conflict detection
• Additional WAL coordination

The API exposes this as “capacity,” but underneath it is really measuring distributed coordination cost.

The Illusion of “The Table”

From a developer’s perspective, DynamoDB exposes a very simple abstraction:

“Here is your table with 1000 RCUs.”

Naturally, you assume that means:

“I can perform 1000 reads per second anywhere in the table.”

But internally, there is no single monolithic table server.

DynamoDB splits the table into multiple physical partitions distributed across storage nodes.

Historically, DynamoDB allocated provisioned throughput evenly across those partitions.

So if:

• Your table had 1000 RCUs
• And the table was split across two partitions

then internally the system looked more like this:

Partition P1 → 500 RCUs
Partition P2 → 500 RCUs

This design implicitly assumed something very important:

Traffic would be distributed relatively uniformly across partition keys.

And that assumption turns out to be one of the hardest problems in distributed databases.

Hot Partitions vs Hot Keys

DynamoDB can automatically split partitions as tables grow.

But there is a very important limitation:

DynamoDB can split partitions. It cannot split individual keys.

Suppose all traffic suddenly targets: PK = USER#123
Every request for that key:

• Must route to the same partition
• Must go through the same replication group
• Must preserve ordering and consistency guarantees

DynamoDB cannot randomly scatter one logical item across multiple machines without fundamentally changing consistency semantics.

This leads to an important distinction:

A Hot Partition means:

• Many different keys collectively overload a partition

A Hot Key means:

• One single key dominates traffic

Partition splitting helps with the first problem.
It cannot fully solve the second.
This is why partition key design matters so much in DynamoDB.

Throughput Dilution

Imagine a table with:

• 1000 RCUs total
• Two partitions

Internally:

P1 → 500 RCUs
P2 → 500 RCUs

Now suppose a very hot key k lives on P2.

Over time:

• P2 accumulates more data
• It reaches storage limits
• DynamoDB decides to split the partition

So P2 becomes:

P21 → 250 RCUs (contains hot key k)
P22 → 250 RCUs

And suddenly something very bad happens.
The maximum throughput available to key k just dropped from:
500 RCUs → 250 RCUs

Why?
Because throughput was evenly redistributed across the child partitions after the split.
So ironically:

Scaling storage capacity accidentally reduced throughput capacity.

This is known as Throughput Dilution.

Eventually, AWS realized strict partition-level admission control created terrible developer experiences.
So DynamoDB evolved.
This evolution introduced:

• Bursting
• Adaptive Capacity
• Global Admission Control

Instead of rigidly limiting every partition independently, modern DynamoDB:

• Allows partitions to temporarily borrow unused capacity
• Dynamically reallocates throughput
• Prioritizes hotter workloads
• Reduces the impact of uneven traffic patterns

This dramatically improved real-world performance and reduced throttling scenarios.
But importantly:

Adaptive Capacity improves hot partition handling. It does not magically eliminate hot key problems.

The physical realities of distributed systems still exist underneath the abstraction.

Conclusion

There is still a lot more depth to DynamoDB internals that we could explore, but I think this is already enough to build the most important mental model: DynamoDB’s API design is not arbitrary. Almost every restriction, cost model, and data modeling recommendation comes directly from the realities of distributed systems and the physical limitations of large-scale storage infrastructure.

Things like partition keys, RCUs/WCUs, eventual consistency, hot partitions, GSI restrictions, transaction costs, and even the way indexes behave all start making much more sense once you look beneath the abstraction layer. DynamoDB feels very different from traditional SQL databases because it is solving a very different problem at a very different scale.

And honestly, that is what makes it fascinating.

Thank you so much for reading through all of this. If you made it this far, I genuinely appreciate it :) If you enjoyed the article or learned something new, do leave a like, comment, and follow. I regularly write about distributed systems, databases, microservices, and other engineering topics that I find interesting.

You can also find me on Twitter, where I occasionally post technical thoughts and occasionally rant about random engineering things too :)

Feel free to check out my other blogs and connect with me on LinkedIn as well.

Thank you again, and take care!