Design Chat Service

A chat service is a real-time messaging platform that enables instant communication between users, like WhatsApp or Facebook Messenger, supporting text, media, voice, and group conversations across devices.

In this blog we will see how we can design a chat service similar to the scale of whatsapp and facebook messenger. Like previous blogs we will start with requirements, then estimations, APIs, Entities and then move to high level design and then finally deep dive the system design.

Requirements

Functional requirements to cover in this blog are:

1. User should be able to send/recieve the messages.
2. User should be able to send/recieve the media files in their messages.
3. User should be able to recieve the messages sent while they were offline.
4. User should be able to start the group chat with multiple participants.

Non-Functional requirements are:

1. Messages should be delivered to online users with low latency.
2. There should be a guarantee of messages deliverability.
3. System should be scalable and efficient to handle high throughput.
4. System should be consistent, messages should appear in the order they were delivered and also there should be no loss of message data.

Entities and Estimations

The core entities in this system will be like

1. Users
2. Chats (2 to multiple members)
3. Messages
4. Devices or clients for each user

We can assume 1 billion DAU with average 50 text messages per day for each user (50 billion). Assuming 20 % of media messages, there will be 10 billion messages. Let’s say on an average 1 KB is the text message size and 10KB is the media message size.

Total storage/day = 1 KB * 50 B + 10 KB * 10 B = 50 TB + 100 TB = 150 TB

Annual storage = 150 TB * 365 = 54750 TB = 55 PB

Considering 20–50 % concurrent users at a time, we need to maintain 200–500K concurrent websocket connections.

Our system is going to be read and write heavy.

APIs

For our chat system, REST APIs are not a good choice, we need bi-directional connection to send and recieve the messages in real time. So we will use websocket here. So the websocket commands will be

Get Adarsh Tyagi’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Commands Sent from user:

1. createChat — to create the new chat
2. sendMessage — to send the messages to chat participants
3. createAttachment — to create media files
4. modifyChatParticipants — to add or remove the participants

Commands recieved by user:

1. chatUpdate — when chat is created or updated
2. newMessage — when a message is recieved

High Level Design

For our initial requirement, we will need to create a chat. We can have a chat service behind the load balancer which can create the chat and write the metadata in database. We can use DynamoDB for key-value lookup and scalability. User will connect to the service and send the createChat command, then service will create the chat record in database along with the record or participants for a chat. We will have a chat table with id, name and metadata and then another table for mapping of participantId with chatId. Simple using primary key on id field of chat table is enough for us to retrive the data and we can have composite primary key on chatId and partitcipantId for chatParticipants table.

Next we need is users to send and recieve the messages. We can make a web socket connection with our chat server. When user makes a web socket connection, we will need to keep record of it, so we map a user with the connection in a simple hash map. To send the message, user will send the sendMessage command to the chat server and then chat server will get the chat participants from the chat participants table we created earlier and then fetch the web socket connection for each participants from its internal hashmap and then send the message.

What if users are not online?

For offline users, we need to store the messages in our database so that we can deliver it later when they comes online. Let’s say we create a inbox for each user. So when a message is sent then it will get stored in the inbox table of each recipient.

1. User will send message by sendMessage command to the chat server.
2. Chat server will fetch all the chat participants.
3. Let’s say we also have a message table which keeps record of all the messages sent by the user. So chat server will run the transaction which have two steps, first it will write the message to the message table and then create the entry in inbox table of each recipient.
4. Chat server will look up the web socket connection of each recipient and run the sendMessage command.
5. For connected clients, server will recieve the ACK command back which means recipient have recieved the message. Then it will delete the message from inbox.
6. Now when recipient comes back online, then chat server will fetch all the undelivered messages from inbox and fetch the content from messages table.
7. Send the messages to the client connection using sendMessage command.
8. And then similar step like (5) will execute.

What about media messages?

Media messages are storage and bandwidth exhaustive. Keeping media attachments in our DB is not optimal as it will use large bandwidth of our chat server and also DBs are not optimal for large binary blobs.

What if chat server recieved the media attachment and then push it to the blob storage like S3 or GCS and keep the metadata in our DB. Now while fetching or reading this data, user will recieve the pre-signe URL of the blob storage but still our chat server have to handle the incoming request and upload it to the storage.

We can allow the user to upload the media directly on blob storage by giving them a pre-signed URL. User can send the createAttachment command to chat server which can return them the pre-signed blob storage URL. After uploading they will recieve the URL of data from blob storage which they can send back to chat server back.

Design Deep Dive

How to handle multiple concurrent users/connections?

For millions and billions of users we will need multiple chat servers. But with multiple chat servers the issue is that it might be possible that sender and reciever are connected with different servers and if this is the case then sender will send the message but recipients will not be able to recieve the messages.

We can use consistent hashing, and assigning the specific server to a specific user. If we do this, then we will know which chat server is repsonsible for a user id and then can send the message directly to it. When request comes in, we can assign the chat server according to user id and when new event is created then chat server will directly connect to the server that owns that user id and then send the message. But the problem is that each chat server needs to maintain connection with each other and if we have too many chat servers then keeping these connection will be a headache.

For every chat among users, we can do the consistent hashing based on each chat id, a chat server is assigned to a chat. So when a user send the message to a chat, it will go to the dedicated chat server and then it will find the websocket connections of recipients present on that chat server and deliver the messsage. But a user can be a part of multiple chats and then it will also be part of multiple servers and the multiple server will have record of web socket connection of each user.

Ok, what about Redis Pub/Sub? So we can use pub/sub which can have a lightweight hashmap of socket connections. With pub/sub you create a subscription for a given user id and then publish the message to that subscription which will recieved by at most once by the subscriber of user id. When user connects to the chat server, chat server will connect to the pub/sub to subscribe to the topic of the user id. Now any message received on that subscription is now forwarded to the user’s socket connection. When a message needs to be sent, we publish message to the topic of relevant user id. The message is recieved by all the subscribed chat servers and then chat servers will send message to the user’s web socket connection. The challenge here is to keep a connection between each chat server and redis pub/sub server.

How to handle multiple clients/devices for a user?

If a user is using the chat service on multiple devices then he wants all his message that were sent on phone should also be present on laptop so that it is in sync. Now this creates some challenges like we need to resolve a user to one or more clients that are active at a time. Second is we need to deactivate the clients so that we do not store the unnecessary message for a client and obviously the message delivery should be able to handle the multiple clients.

We can create a clients table mapped with each user id. While looking up for a participant of a chat, we will also look up for all the client id for that participant. Also we can update our Inbox table to be for client id rather than user id. When we need to send the message, we will send message to each client of the user id. To avoid unnecessary storage and throughput we can enforce some limits like upto 3 clients for each user.

Architecture Diagram

Now let’s see the final architecture diagram that we have

Char Service Design Diagram