Why Are Python Logs Jumbled In Cloudwatch?

Have your print statements or logs ever appeared in a completely unexpected order?

If you’ve ever worked with a separate logging service like AWS CloudWatch, at some point your logs must have been printed in random chunks in random places. I’ve gone down the rabbit hole, testing and figuring out why and when this happens, the many ways it can occur, and how to avoid it. In this blog, I have condensed all the insights I have gathered.

For non-members, click here to read the blog:

Buffering in Action: The Curious Case of Jumbled Logs

Following is a small piece of code that uses multiple types of logs. In what order is it going to print the logs?

# test.py
import sys
import logging

logging.basicConfig(level=logging.INFO)

logging.info("This is the start")
print("This is standard output") # print statments writes to standard output in general
print(f"This is standard error", file=sys.stderr) # this is writing to stderr
logging.info("This is the end")

When you run it on the terminal, with `python test.py`. You get the following output which is as expected.

$ python test.py
INFO:root:This is the start
This is standard output
This is…