Concurrency

Concurrency is what happens if multiple things try to happen at same time.

Examples:

• Parking Lot → Two cars try to occupy the last slot.
• BookMyShow → Two users book the same seat.
• Elevator → Multiple requests arrive simultaneously.
• Rate Limiter → Hundreds of requests update the same counters.
• Cache → Multiple threads refresh the same key.
• Logger → Multiple threads write logs.

Low-level design interviews focus on single-process systems, so the concurrency here is all threads and shared memory within one program. System design interviews deal with concurrency across multiple servers — that’s a different beast.

Concurrency Fundamentals

Concurrency starts with a basic fact: threads in the same process share memory.

Three Problem Types

Most concurrency problems in interviews fall into three categories. The surface details change — inventory systems, booking systems, rate limiters — but the underlying failure modes don’t.

Correctness problems happen when shared state gets corrupted. Two threads both check that a seat is available, both see yes, both book it. One booking gets lost.

Coordination problems happen when threads need to hand off work or wait for each other. A producer adds tasks to a queue, consumers process them. If the queue is empty, consumers need to wait efficiently without burning CPU. If the queue is full, producers need to slow down.

Get Sahil Kalra’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Scarcity problems happen when resources are limited. You have 10 database connections but 100 concurrent requests. Some requests must wait.

Correctness

Shared state gets corrupted because two threads access it at the same time.

// check-then-act

public class BookingService{
   Map<String, Seat> seats;

   public boolean bookSeat(String seatId, String visitorId){
      Seat seat = seats.get(seatId);
      if(seat.isAvailable()){
          // this is the trouble area
          seat.setOccupant(visitorId);
          return true;
      }
      return false;
   }

    // THE FIX: lock covers both CHECK and Action
    public boolean bookSeat(String seatId, String visitorId){
      Seat seat = seats.get(seatId);
      synchronized(lock){
        if(seat.isAvailable()){
          // this is the trouble area
          seat.setOccupant(visitorId);
          return true;
        }
        return false;
      }
   }
}


// read-modify-write
private int count = 0;

public void increment(){
    count++; // This is actually read count, add 1, write count
}

private AtomicInteger count = new AtomicInteger(10);
public void increment(){
    count.incrementAndGet();
}

Coordination

Thread passing work to each other efficiently

When a producer adds task to the queue, Consumer process them. If the queue is empty, consumers need to wait efficiently without burning CPU. If the queue is full, producers need to slow down.

Two main issues that occur are:

1. How does consumer know that work arrived ?
2. What happens if the work is arriving too fast for the consumer to keep up ?

// BAD
while(true){
    if(!queue.isEmpty()){
        Task task = queue.poll();
        process(task);
    }
    // spins forever when the queue is empty
}

// Also Bad: sleep-polling adds latency
while(true){
    if(!queue.isEmpty()){
        Task task = queue.poll();
        process(task);
    }else{
       Thread.sleep(100); // 100ms latency on every task
    }
}

BlockingQueue<Task> queue = new LinkedBlockingQueue<>(1000);

// Producer - API Handler
public void handleSignUP(User user){
    saveUser(user);
    queue.put(new EmailTask(user)); // blocks if queue is full
    return success();
}

// Consumer - worker thread
while(true){
  Task task = queue.take(); // blocks if queue is empty, zero CPU
  process(task);
}

Scarcity

Limiting concurrent access to finite resources

Semaphore permits = new Semaphore(5); // max 5 concurrent

public void download(String url){
    permits.acquire();
    try(){
      download(url);`
    } finally(){
        permits.release();
    }
}

When managing actual objects with state, create a pool with blocking queue

// Initialize pool with connections
BlockingQueue<Connection> pool = new LinkedBlockingQueue<>();

for(int i=0;i<10;i++) pool.put(createConnection());

// Usage
public Result query(String sql){
  Connection conn = pool.take(); // blocks if queue is empty

  try(){
    return conn.execute(sql);
  } finally(){
       pool.put(conn);
  }
}