Distributed Locks

What is Distributed Lock?

A distributed lock is a mechanism that ensures only one process across multiple machines/services can access or modify a shared resource at a time.

Think of it as a mutex that works across servers.

What are you using that lock for?

The purpose of a lock is to ensure that among several nodes that might try to do the same piece of work, only one actually does it (at least only one at a time). That work might be to write some data to a shared storage system, to perform some computation, to call some external API, or suchlike. At a high level, there are two reasons why you might want a lock in a distributed application: for efficiency or for correctness [2]. To distinguish these cases, you can ask what would happen if the lock failed:

• Efficiency: Taking a lock saves you from unnecessarily doing the same work twice (e.g. some expensive computation). If the lock fails and two nodes end up doing the same piece of work, the result is a minor increase in cost (you end up paying 5 cents more to AWS than you otherwise would have) or a minor inconvenience (e.g. a user ends up getting the same email notification twice).
• Correctness: Taking a lock prevents concurrent processes from stepping on each others’ toes and messing up the state of your system. If the lock fails and two nodes concurrently work on the same piece of data, the result is a corrupted file, data loss, permanent inconsistency, the wrong dose of a drug administered to a patient, or some other serious problem.

Both are valid cases for wanting a lock, but you need to be very clear about which one of the two you are dealing with.

For Efficiency

Redis

If using for Efficiency, you can use a single Redis instance, of course you will drop some locks if the power suddenly goes out on your Redis node, or something else goes wrong. But if you’re only using the locks as an efficiency optimization, and the crashes don’t happen too often, that’s no big deal. This “no big deal” scenario is where Redis shines. At least if you’re relying on a single Redis instance, it is clear to everyone who looks at the system that the locks are approximate, and only to be used for non-critical purposes.

How it works ?

• SET key value NX PX 30000 — atomic set-if-not-exists with expiry
  Where:
  NX = only set if key doesn’t exist
  PX = expiry time
• Release via Lua script checking ownership before delete (avoid deleting someone else’s lock)

if redis.call("GET", key) == value then
   return redis.call("DEL", key)
end

Pros:

• Fast
• Simple

Cons:

• Single Redis instance can become a SPOF.

Redlock

If you are using locks merely for efficiency purposes, it is unnecessary to incur the cost and complexity of Redlock, running 5 Redis servers and checking for a majority to acquire your lock. You are better off just using a single Redis instance, perhaps with asynchronous replication to a secondary instance in case the primary crashes.

How it works ?
Uses:

• Multiple Redis nodes
• Acquire lock on a majority of N independent Redis instances within a time budget

Example:

Redis1 -> success
Redis2 -> success
Redis3 -> success
Redis4 -> fail
Redis5 -> fail

3/5 majority → lock acquired.

Pros:

• Better fault tolerance than single Redis Node

Cons:

• Some distributed systems experts criticize its guarantees under network partitions.

Fencing Tokens

Problem:

Service A gets lock
Network pause
Lock expires
Service B gets lock
Service A resumes

Now both think they own the lock.

Solution:

• Monotonically increasing token.

A -> token 100
B -> token 101

Database accepts only highest token.

This prevents stale lock holders from corrupting data.

For Correctness

etcd

etcd is a distributed, strongly consistent key-value store designed for storing critical cluster state and coordinating distributed systems.

Why etcd Exists

Imagine you have 10 application servers and need to store:

• Who is the leader?
• Which jobs are running?
• Service discovery information
• Distributed locks
• Configuration data

A normal cache like Redis prioritizes speed. etcd prioritizes:

✅ Consistency
✅ Reliability
✅ Fault tolerance

ETCD Lock

etcd’s lock isn’t a special primitive — it’s built from two core pieces:

1. Leases: a TTL attached to a key. The client must send periodic keepalives; if it stops (crash, network partition, long pause), the lease expires and etcd auto-deletes the associated key.
2. Raft consensus: every write (including lease creation/expiry) goes through Raft, so all etcd nodes agree on lock state — this is what gives you strong consistency, unlike Redlock’s quorum-of-independent-instances approach.

Internally: Lock() creates a key like /my-lock/<lease-id> and waits until it has the lowest revision among keys with that prefix (similar idea to ZooKeeper's sequential znodes). Other waiters watch the key ahead of them and get woken when it's deleted (released or lease expired).

Fencing tokens with etcd (important)

Every key in etcd has a mod_revision — a global, strictly increasing number. Use this as your fencing token:

This protects you against the classic failure mode: your client’s lease expires (maybe due to a GC pause or slow network) while it still believes it holds the lock, and it tries to act after another client has already acquired the lock. The downstream resource rejects the stale write.

Zookeeper Lock

How it works ?

Using:

• Ephemeral Sequential Nodes

Example:

/lock/lock-0001
/lock/lock-0002
/lock/lock-0003

Smallest sequence owns lock.

If owner dies:

• Ephemeral node disappears.
• Next node automatically gets lock.

Pros:

• Strong consistency

Used by:

• Hadoop ecosystem
• Kafka (historically)

Why Not Redis For Correctness usecases?

Redis locks are generally suitable when:

• Duplicate execution is acceptable occasionally
• Lock is used for efficiency rather than correctness

Example:

Avoid running same report twice

Not ideal when:

Prevent selling same ticket twice
Prevent double payment
Prevent inventory corruption

Because network partitions, clock issues, and lock expiration races can cause multiple clients to believe they own the lock.

Why not Redlock here

Redlock (multi-Redis-instance) gives you availability and decent practical safety, but it’s not built on a real consensus algorithm — it relies on timing assumptions (bounded clock drift, no long process pauses) and it does not support fencing tokens.

Raft Consensus vs Redlock’s quorum of independent instances

At first glance, both Raft and Redlock seem to use a “majority of nodes” idea, but they solve fundamentally different problems.

Raft is a consensus algorithm.

Goal:

Ensure every node agrees on exactly the same sequence of operations.

Example:

Cluster:
A (Leader)
B (Follower)
C (Follower)

Client writes:

lock = user123

Flow:

1. Leader receives write
2. Leader appends to log
3. Followers replicate log
4. Majority ACKs
5. Commit

Result:

A log: [lock=user123]
B log: [lock=user123]
C log: [lock=user123]

All nodes eventually converge to the same state and same ordering.
Redlock

Redlock is not consensus.

It is a lock acquisition protocol built on multiple independent Redis instances.

Example:

Redis1
Redis2
Redis3
Redis4
Redis5

Client attempts:

SET lock NX PX

on all 5 nodes.

Get Sahil Kalra’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

If it succeeds on 3/5:

R1 = success
R2 = success
R3 = success
R4 = fail
R5 = fail

Lock considered acquired.

Key difference

The Redis nodes do not replicate state to each other.

There is no shared log.

There is no leader.

There is no agreed ordering.

Each Redis instance acts independently.

What Do Large Companies Use?

For correctness-critical workflows, many companies avoid relying solely on distributed locks and instead use:

Database Transactions + Row Locks + Unique Constraints

This often provides stronger guarantees than an external lock service.

For ticket booking systems, payment systems, and inventory management:

PostgreSQL Transaction
+
Row Lock
+
Unique Constraints

is frequently preferred over Redis locks.

Row Lock (SELECT FOR UPDATE)

Imagine table:

Seats
-----
seat_id | status
A1      | AVAILABLE

Transaction A:

BEGIN;
SELECT *
FROM seats
WHERE seat_id='A1'
FOR UPDATE;

Database:

Locks row A1

Now Transaction B:

BEGIN;
SELECT *
FROM seats
WHERE seat_id='A1'
FOR UPDATE;

Database:

WAIT

because A already owns the row lock.

Timeline

T1: Transaction A locks row
T2: Transaction B waits
T3: Transaction A updates status=BOOKED
T4: Transaction A commits
T5: Transaction B resumes

Now B sees:

A1 = BOOKED

and fails booking.

Result:

Only one booking

Unique Constraints

Even stronger.

Table:

CREATE TABLE bookings (
    booking_id BIGINT,
    seat_id VARCHAR(20)
);

ALTER TABLE bookings
ADD CONSTRAINT unique_seat
UNIQUE(seat_id);

Now database guarantees:

One seat -> One booking row

Concurrent Insert Example

Transaction A:

INSERT INTO bookings
VALUES (1,'A1');

Transaction B:

INSERT INTO bookings
VALUES (2,'A1');

Both run simultaneously.

Database:

Transaction A succeeds

Transaction B
↓
Unique Constraint Violation

Result:

Only one booking exists

No distributed lock needed.

Database Transaction

A transaction groups operations into one atomic unit.

Example:

BEGIN;
Check seat
Create booking
Charge customer
Update seat status
COMMIT;

Either:

Everything succeeds

or

Everything rolls back

How Database Acts Like a Distributed Lock

Imagine:

App Server A
App Server B
App Server C

All connect to PostgreSQL.

When A executes:

SELECT ... FOR UPDATE

PostgreSQL stores lock metadata internally.

PostgreSQL
   |
 Lock Table
   |
Seat A1 -> Txn 123

Now every server sees:

Seat A1 locked

because they all use the same database.

The database becomes the central coordinator.

Optmistic Concurrency Control (OCC)

Optimistic Concurrency Control (OCC) is one of the most important alternatives to distributed locking.

The core idea is:

Don’t lock first. Assume conflicts are rare. Detect conflicts when writing.

OCC Solution

Add a version column:

Products
---------
id
price
version

Current state:

Price = 100
Version = 1

User A reads:

Price = 100
Version = 1

User B reads:

Price = 100
Version = 1

User A updates:

UPDATE products
SET price = 120,
    version = 2
WHERE id = 1
  AND version = 1;

Rows affected:

1

Success.

User B updates:

UPDATE products
SET price = 130,
    version = 2
WHERE id = 1
  AND version = 1;

Rows affected:

0

because version is now 2.

Database rejects the update.

When OCC Works Well

Read-heavy systems

10000 reads
10 writes

Conflicts are rare.

Examples:

• Product catalog updates
• User profile updates
• Content management systems
• Configuration systems

OCC scales extremely well because there are no locks.

Why do we still Need to use something like Redis, etcd for Distributed Locks

Consider a ticket booking system where two users, User1 and User2, attempt to book the same seat (Seat A) at nearly the same time. If we rely solely on database transactions and row-level locks, the system can prevent double booking, but the user experience may still be poor. Both users may be allowed to select the seat and proceed through the booking flow. One user could spend time entering payment details and completing the payment process, only to discover at the final step that the seat was already booked by someone else.

To avoid this scenario, we can introduce a distributed lock (for example, using etcd) to implement a temporary seat reservation mechanism. When a user selects a seat, the system acquires a lock and reserves that seat for a limited duration (e.g., 30 seconds). During this reservation window, other users cannot select or reserve the same seat. The user can then complete the payment and confirm the booking. If the payment is not completed within the TTL, the lock expires automatically, and the seat becomes available for others to book.

This approach not only prevents double booking but also provides a much better user experience by reducing the likelihood of users investing time in a booking flow only to be rejected at the end