Understand I/O Multiplexing and Asyncio Streams work in Python
This post is a written companion to my talk at PyCon India 2025. I could not complete my talk in time and hence this post is a complete version of it.
What Is I/O Multiplexing?
Multiplexing is handling multiple things over a single shared resource. In telecom, it’s multiple phone calls over one wire. In networking, it’s multiple data streams over one connection.
I/O multiplexing applies this to a program: a single thread monitoring multiple I/O sources: sockets, files, pipes, and reacting to whichever one is ready. Instead of dedicating one thread to each connection and having most of them sit idle, you have one thread that says, “tell me when any of these has data,” and then handles whichever ones are ready.
The operating system provides the machinery for this: system calls like select, poll, epoll (Linux), and kqueue (macOS) that let your program hand the kernel a list of file descriptors and say, "wake me up when something happens on any of these."
Why is I/O multiplexing even relevant to us?
Your socket server calls recv() and blocks: while it's waiting on one client, every other client is invisible. One fix is a thread per connection, but that does not scale; threads carry memory overhead (~8 MB stack each), OS context-switching costs, and Python's GIL restricts one thread to run Python bytecode at a time. With I/O Multiplexing, a single thread can handle multiple connections for a fraction of the cost. Asyncio is a popular library to do this in modern python, which this will post will dig into later.
Sockets Are File Descriptors
When your program opens a socket connection, it doesn’t interact with the network interface directly. Instead, the kernel gives it a file descriptor, and your program reads from and writes to that file descriptor while the kernel handles the networking underneath.
Similarly, file descriptors 0, 1, and 2 in any program are your standard input, standard output, and standard error. A socket is just another file descriptor, maybe number 5 or 6 or 42.
The kernel provides tools to monitor file descriptors for readiness and those tools work for sockets too.
Monitoring File Descriptors with select
The select syscall lets you hand the kernel a list of file descriptors and say, “tell me when any of these are ready to read from or write to.” Your program then blocks until something is ready (or until the timeout expires, if you set one) and handles whatever is ready.
Here’s an example using select: a non-blocking way to read from standard input:
import select
import sys
while True:
ready, _, _ = select.select([sys.stdin], [], [], 0) # timeout=0, non-blocking
if ready:
data = sys.stdin.readline()
print(f"Got: {data.strip()}")
# ... do other work hereCompare this to input(), which blocks your program until the user types something. With select, your program can check whether stdin has data, and if it doesn’t, do something else.
select can monitor multiple file descriptors at once. Say you have a socket connection and standard input, you can tell select to watch both.(comment if you know of a different way to achieve the same :D)
import sys
import select
import socket
client = socket.socket()
client.connect(("localhost", 8888))
while True:
readable, _, _ = select.select([sys.stdin, client], [], [])
if readable:
for reader in readable:
if reader is client:
data = client.recv(1024)
print(f"Server >>> {data.decode()}")
else:
stdin_message = sys.stdin.readline()
client.sendall(stdin_message.encode())This is a simple chat client, you can send multiple messages to the server and receive messages from it without each operation blocking each other, all in a single thread. If you’d used input(), you'd be stuck waiting for keyboard input and would never receive incoming server messages until you typed something.
This is I/O multiplexing in its simplest form: one thread, multiple file descriptors, acting on whichever is ready.
(A quick note: select can only monitor up to 1024 file descriptors. Python's selectors module picks the best OS-level mechanism automatically:epoll on Linux, kqueue on macOS which scales much better)
Building a Select-Based Server
Let’s build a server using select that handles multiple clients:
import socket
import select
server = socket.socket()
server.bind(("localhost", 8888))
server.listen()
wait_read = [server]
while True:
readers, _, _ = select.select(wait_read, [], [])
for reader in readers:
if reader is server:
client_sock, addr = server.accept()
print(f"Got new connection from {addr}")
wait_read.append(client_sock)
else:
data = reader.recv(1024)
if not data:
wait_read.remove(reader)
reader.close()
reader.send(large_data) # send a large responseHere’s how it works: we register the server socket with select and wait. When select returns, we loop over the ready sockets, and each one falls into one of two cases. If it’s the server socket, “ready” means a new client is trying to connect, so we accept() it and add the new client to our watch list. If it’s a client socket, “ready” means the client sent data, so we read it and send back a large response.
The problem: when we call reader.send(large_data), that call blocks until the entire payload is written. While the thread is busy sending data to client one, client two is waiting.
The Problem
Connect two clients to this server. Client one sends a message, triggering the server to stream the large payload back. Client two sends a message too. But client two gets nothing; it’s completely blocked while client one receives all the data.
The thread is stuck on Client A’s send(). Client B doesn't exist until A is done.
This can be an undesirable effect and lets understand why this is happening.
Get Rohan Reddy Alleti’s stories in your inbox
Join Medium for free to get updates from this writer.
The answer lies in socket buffers.
How Socket Buffers Work
send() doesn't write directly to the network, it writes into a kernel-managed buffer, which has a fixed size. When that buffer is full, send() blocks until the kernel flushes some of it onto the network. With 1 MB of data and a 10 KB buffer, the thread is stuck in a loop: write 10 KB, wait for flush, write 10 KB, wait for flush; completely occupied with one socket while every other client waits.
Non-Blocking Sockets
conn.setblocking(False) changes send() fundamentally: it writes as much as the buffer can hold and returns immediately, even if it didn't send all your data. It returns the number of bytes it actually wrote.
n = conn.send(large_data) # might only send 10KB out of 1MB
# n tells you how many bytes were actually sent.
# the rest? that needs to be tracked and re-sent.Same two clients, same large payload; but now both receive data, chunk by chunk:
Both clients get data in chunks. Neither blocks the other.
With non-blocking sockets, one thread can serve many clients by reading and writing small chunks of data. While the kernel flushes Client A’s buffer to the network, the thread writes to Client B’s buffer. When A’s buffer has room again, the thread comes back and writes more. The rhythm is: read a little, write a little, buffer full, move on, come back later. No client gets the thread’s full attention, so each one’s throughput is lower, but every client makes progress and the server stays responsive to all of them. That’s the tradeoff, and for almost every network application, it’s exactly what you want.
This is also fundamentally how tools like Redis achieve their performance: a single thread, non-blocking I/O, never wasting a cycle waiting on a socket.
The Developer’s Burden
The tradeoff: you now have to track partial sends yourself. If send() only writes 2 KB out of 10 KB, those remaining 8 KB are your responsibility. Here's the non-blocking version of our server:
import socket
import select
from pathlib import Path
file = Path("smallfile.txt")
large_data = file.read_bytes() * 1000 # big enough that send() can't push it all at once
server = socket.socket()
addr = ("localhost", 8888)
server.bind(addr)
server.listen()
read_wait = {server} # sockets select() should watch for incoming data
write_wait = set() # sockets we still owe data to
send_buffers = {} # socket -> bytes not yet sent (the bookkeeping)
while read_wait:
# block until at least one socket is ready to read or write
readers, writers, _ = select.select(read_wait, write_wait, [])
for reader in readers:
if reader is server:
# new client knocking
client_sock, addr = server.accept()
client_sock.setblocking(False) # send/recv return immediately, never stall
print(f"Got new connection from {addr}")
read_wait.add(client_sock)
else:
data = reader.recv(1024)
if not data:
# empty recv = client closed the connection
read_wait.remove(reader)
reader.close()
else:
# queue the reply instead of sending it right now
send_buffers[reader] = large_data
write_wait.add(reader)
for writer in writers:
data_to_send = send_buffers.pop(writer)
sent = writer.send(data_to_send) # non-blocking: may send only PART of it
if sent < len(data_to_send):
# put the leftover back; select() will tell us when to continue
send_buffers[writer] = data_to_send[sent:]
else:
# all sent, stop watching this socket for writes
write_wait.remove(writer)We now have send_buffers, write_wait, and we're tracking how many bytes were actually sent. If send() only sent a portion of the data, we slice off what was sent and store the remainder back in send_buffers so we can come back to it later. We're also using select with both read_wait and write_wait now: we need to know when a socket is ready for writing, not just reading.
It works, but it’s a lot of bookkeeping. And this is a simple echo server. Imagine doing this for a real application with multiple message types, connection state, timeouts, and error handling. This is where asyncio comes in.
Asyncio Streams: The Whole Package
Asyncio uses non-blocking sockets under the hood. It uses the same select/epoll/kqueue mechanisms we've been talking about. But it handles all the painful parts for you:
import asyncio
from asyncio import StreamWriter, StreamReader
from pathlib import Path
file = Path("smallfile.txt")
large_data = file.read_bytes() * 1000 # Simulating large data
async def handle_echo(reader: StreamReader, writer: StreamWriter):
while True:
data = await reader.read(1024)
if not data:
break
writer.write(large_data)
# await writer.drain()
writer.close()
await writer.wait_closed()
async def main():
server = await asyncio.start_server(handle_echo, '127.0.0.1', 8888)
addr = server.sockets[0].getsockname()
print(f"Serving on {addr}")
async with server:
await server.serve_forever()
asyncio.run(main())Same large_data, same behavior, but no send_buffers dict, no write_wait set, no manual byte slicing. Asyncio's streams API handles all of that internally. It maintains an application-level buffer, keeps track of what's been sent and what hasn't, and automatically writes data into socket buffers when they're ready.
Notice that await writer.drain() is commented out. We'll come back to why.
What await Actually Does
When the coroutine hits await reader.read(1024), it doesn’t block the thread. Instead, it’s saying: “I have nothing to do until data arrives on this socket, so go run something else.” The asyncio event loop takes over, checks which other sockets are ready, runs the coroutines waiting on them, and returns to this coroutine once its data arrives.
This is cooperative scheduling: coroutines voluntarily give up control at await points. Threads work differently. The OS preemptively switches between them whenever it decides to, and each switch is a full context switch, which is far more expensive than resuming a coroutine. The flip side of cooperation is that a coroutine that never awaits, such as one stuck in a long computation, holds up every other coroutine on the loop.
A Note on writer.drain()
Remember the commented-out await writer.drain() in the code above? If you run the server without it, everything works and both clients receive their data. So why bother?
In our select server, we had to keep track of any data that wasn’t fully sent after a partial send() and try again later. Asyncio does that bookkeeping for you. writer.write() doesn’t write to the socket directly; it appends to asyncio’s internal buffer, and asyncio flushes that buffer into the kernel’s socket buffer in the background, handling partial sends and tracking what’s been sent. (In recent versions of CPython, when a send() only partially succeeds, asyncio holds onto the unsent portion as a memoryview, a lightweight reference to the remaining slice, rather than copying those bytes into a new buffer.)
The catch every write() call appends to the buffer, no matter how much is already waiting. If your code produces data faster than the network can send it, the buffer keeps growing, and your server’s memory grows silently with it until the process runs out.
That’s where drain() comes in. When you await writer.drain(), asyncio checks how much data is still waiting in the buffer. If it’s grown too large, your coroutine pauses until enough of it has been flushed to the network, then resumes. Instead of piling data into the buffer forever, your code slows down to match what the network can handle. This is called backpressure.
What’s Next: Free-Threaded Python
The free-threaded build of Python (no GIL) arrived experimentally in Python 3.13 and became officially supported in 3.14, though it’s still opt-in rather than the default. Without the GIL, threads can finally run Python code truly in parallel across multiple CPU cores.
That doesn’t mean you should switch from asyncio to one thread per connection. If your workload is I/O-bound, managing ten thousand connections with ten thousand threads, even without the GIL, still means ten thousand stack allocations, ten thousand entries for the OS scheduler to juggle, and a context switch every time a thread blocks on a socket. The overhead of managing the threads can dwarf the actual work being done.
Where free threading gets interesting is in combining the two approaches. Instead of one event loop on one core, you can run one thread per CPU core, each with its own event loop handling thousands of connections. Each thread does I/O multiplexing exactly as we’ve seen, but now they all run in parallel, so the server can use every core instead of just one.
Wrapping Up
If there’s one thing I hope you take away, it’s this: asyncio isn’t magic. It’s an event loop built on select/epoll/kqueue, non-blocking sockets, and careful buffer management.
The next time you write await writer.drain(), you’ll know exactly which buffer it’s waiting on, and why.
This post is based on my talk at PyCon India 2025, watch it here if you want the live demos.
