Sitemap
Shariful Islam Sharif

Shariful Islam Sharif

I am a software engineer specializing in Python, Django, and Elasticsearch, now expanding my skills into DevOps to focus on infrastructure and automation.

Press enter or click to view image in full size
Image copied from https://geekpython.in/

Threading in Python

10 min readJan 18, 2025

--

What is thread?

According to Wikipedia “In computer science, a thread of execution is the smallest sequence of programmed instructions that can be managed independently by a scheduler, which is typically a part of the operating system. In many cases, a thread is a component of a process.”

In simple terms, a thread is like a separate flow of execution within a computer program. Imagine a program as a single kitchen. A thread is like a single chef working within that kitchen.
A process is like the entire kitchen itself, with all its equipment and resources. It is an independent execution environment. Each process has its own memory space, resources, and a dedicated execution path.
A thread is like an individual chef within that kitchen. Multiple chefs can work independently within the same kitchen, sharing some resources (like ingredients, equipment etc.) but having their own tasks and work areas.

If we wanted to look at a real scenario then we can assume, A program like Microsoft word is a process and inside this word program, running sub-task like searching a text, using find and replace, spelling mistake, grammar mistake is thread. A process you can understand as an independent & active program and a thread is a lightweight process and a part of main process which share memory with other thread and use same resources of process.

Press enter or click to view image in full size
Photo is downloaded from wikipedia.

Key Characteristics

  • Lightweight: Threads within the same process share memory, making them more efficient than creating multiple processes.
  • Concurrency: Threads can run different parts of a program concurrently, improving responsiveness.
  • Shared Resources: Threads can easily share data and resources since they exist within the same process, but this requires careful management to avoid conflicts.

How to use Threads in Python?

Threads in Python allow you to run multiple threads (smaller units of a process) concurrently within a single process. Python provides the threading module to work with threads easily.threading module provides tools for creating and managing threads.

  • Thread class: You can create and manage threads using the Thread class.
  • start() method: Starts the execution of a thread.
  • join() method: Waits for a thread to complete execution.

Create and Start a Thread

To create a new thread, we create an object of the Thread class. It takes the target and args as the parameters. The target is the function to be executed by the thread whereas the args is the arguments to be passed to the target function. And then to start a thread, we use the .start()method of the Thread class.

import threading
import time

def test_method():
print("Method started executing.....")
time.sleep(3)
print("Method executed successfully.")

if __name__ == "__main__":
thread = threading.Thread(target=test_method) # Create a thread

print("Starting a thread.")
thread.start() # Start the thread
print("Thread Finished.")

"""
Sample Output

Starting a thread.
Method started executing.....
Thread Finished.
Method executed successfully.
"""

That is how we can create and start a thread in python. Wait! did you notice any inconsistency in the output? I think you did. Lets discuss why and how we can fix it.

Why the inconsistency: The thread starts executing concurrently with the main program and the main program does not wait for the thread to finish before continuing its execution. That’s why the above code resulted in executing the print statement before the thread was finished.

To understand this, you need to understand the execution flow of the program:

  • First, the "Starting a Thread." print statement is executed.
  • Then the thread is created and started using thread.start().
  • The thread starts executing concurrently with the main program.
  • The "Thread Finished." print statement is executed by the main program.
  • The thread continues and prints the output.

The thread and the main program run independently that’s why their execution order is not fixed.

Seeing the above situation, you might have thought then how to suspend the execution of the main program until the thread is finished executing.
Well, the join() method is used in that situation, it doesn’t let execute the code further until the current thread terminates.

What join() method does?

The join() method in Python's threading module is used to make the main program (or the calling thread) wait until the thread it is called on has completed its execution. In simple terms, it ensures that the thread finishes before the program continues.
It helps synchronize threads, ensuring one thread completes before the next step in the program. Without join(), the main program may finish and exit before threads complete their tasks.

def test_method():
print("Method started executing.....")
time.sleep(3)
print("Method executed successfully.")

if __name__ == "__main__":
thread = threading.Thread(target=test_method) # Create a thread

print("Starting a thread.")
thread.start() # Start the thread
thread.join()
print("Thread Finished.")

# Sample Output
"""
Starting a thread.
Method started executing.....
Method executed successfully.
Thread Finished.
"""

So when we are not using join() method, the main program is not waiting for the thread(child thread in this case) to finish.

Using join() with Timeout

We can specify a optional parameter timeout to wait for the thread to complete for a certain amount of time by thread.join(timeout=2) .
If the thread completes within 2 seconds, the program continues normally. If it doesn’t, the program resumes without waiting for the thread to finish. It has following benefits

  • Prevent indefinite blocking while waiting for a thread to finish.
  • Avoid deadlocks caused by stuck threads.
  • Handle long-running threads gracefully and implement fallback logic.
  • Balance between waiting for thread completion and maintaining responsiveness.

How to pass parameters in thread?

We can pass different parameters to a thread by using the args and kwargs arguments when creating a thread. These allow you to pass positional and keyword arguments, respectively, to the target function.

# Create a thread and pass positional arguments
import threading

def print_values(a, b, c):
print(f"a: {a}, b: {b}, c: {c}")

thread = threading.Thread(target=print_values, args=(1, 2, 3))
thread.start()
thread.join()

# Create a thread and pass keyword arguments
def print_values(x, y, z):
print(f"x: {x}, y: {y}, z: {z}")

thread = threading.Thread(target=print_values, kwargs={'x': 10, 'y': 20, 'z': 30})
thread.start()
thread.join()

What is Daemon thread?

In computer science, a daemon is a process that runs in the background. Python threading has a more specific meaning for daemon. A daemon thread will shut down immediately when the program exits.
Daemon threads run in the background and terminate immediately whether they completed the work or not when the main program exits.
In other words, daemon thread does not block main thread to exiting and continue run in the background.

Get Shariful Islam Sharif’s stories in your inbox

Join Medium for free to get updates from this writer.

So a daemon thread runs in the background, stops automatically . All the threads are not daemon threads unless explicitly set. We can make a thread a daemon thread by setting its daemon attribute to True before starting it or can pass daemon=True in the parameter.

def test_daemon_thread():
print("Demon thread started running...")
time.sleep(1)
print("Demon thread completed.")


if __name__ == "__main__":
thread1 = threading.Thread(target=test_daemon_thread, daemon=True)

print("Starting a thread.")
thread1.start()
print("Main program Finished.")

# Sample Output
"""
Starting a thread.
Demon thread started running...
Main program Finished.
"""

At the time when the test_daemon_thread() function enters the loop, the concurrently running main program exits, and the test_daemon_thread() function never reaches the next print statement as can be seen in the output.

Lets discuss some use cases of daemon Threads

  • Background Tasks: Daemon threads are ideal for running tasks that are not critical to the main program’s operation, such as garbage collection. For example, you can run a thread to clear unused memory in the background while your application is open. When the application is closed, the background thread should not prevent the main thread from exiting and will terminate without disrupting the program’s shutdown process
  • Monitoring and Logging: Daemon threads are useful for observing system states or logging events while the program runs. For instance, if you need to monitor a log file and take action whenever a specific pattern is detected, you can create a daemon thread to handle the monitoring in the background. Meanwhile, the main thread can continue executing other tasks. This allows the program to exit cleanly even if the log monitoring thread is still running..
  • Periodic task execution: For tasks that need to be performed at regular intervals, a daemon thread can execute them on a schedule. This ensures that the task continues to run periodically, even if the main thread has finished executing.
  • Batch Processing: Daemon threads can be helpful for handling resource-intensive processes, such as image processing in an API. For example, if you need to process a large number of images, you can use a daemon thread to handle the processing in the background. This allows you to return an initial response to the user while the processing continues. Once the processing is complete, you can send the final results to the user via a callback URL.

Thread Synchronization and Locking

Thread locking in Python refers to a mechanism used to prevent multiple threads from accessing/modifying shared resources (e.g., variables, files, or data structures) simultaneously, which could lead to race conditions or inconsistent data states. Python provides the Lock class in the threading module for this purpose.

A Lock is an object that acts like a hall pass. Only one thread at a time can have the Lock. Any other thread that wants the Lock must wait until the owner of the Lock gives it up.

The basic functions to do this are .acquire() and .release(). A thread will call my_lock.acquire() to get the lock. If the lock is already held, the calling thread will wait until it is released.

Lets talk about a meeting room of an office. At any point of time only one person can occupy that meeting room. Until and unless that person comes out of the room, other person can’t go in. Two persons wanted to access the meeting room at the same time. Lets say P1 is currently accessing the meeting room. So P2 should wait until P1 finished. How can fix it by threading?

def first_person():
print("First person booked the room.")
time.sleep(1)
print("first person vacates the room.")

def second_person():
print("Second person booked the room.")
time.sleep(1)
print("Second person vacates the room.")


if __name__ == '__main__':
thread1 = Thread(target=first_person)
thread2 = Thread(target=second_person)

thread1.start()
thread2.start()

thread1.join()
thread2.join()

"""
Sample Output
First person booked the room.
Second person booked the room.
first person vacates the room.
Second person vacates the room.
"""

According to the output, we can see that two person is accessing the room at the same time, which is not possible. We can think of a database or file . Only one person can write databases at any point of time.
Here synchronization comes into action and this behavior is common for all programming languages not only in python. We don’t know which thread will execute when and which purposes. We have to handle the situation. We can mitigate this problem by using Lock in python

def first_person_with_lock(lock):
lock.acquire()
print("First person booked the room.")

time.sleep(1)
print("First person vacates the room.")
lock.release() # task finished lock released.

def second_person_with_lock(lock):
lock.acquire()
print("Second person booked the room.")

time.sleep(1)
lock.release() # task finished lock released.
print("Second person vacates the room.")

if __name__ == '__main__':
lock = threading.Lock()

thread1 = Thread(target=first_person_with_lock, args=(lock,))
thread2 = Thread(target=second_person_with_lock, args=(lock,))

thread1.start()
thread2.start()

thread1.join()
thread2.join()

"""
Sample Output

First person booked the room.
First person vacates the room.
Second person booked the room.
Second person vacates the room.
"""

So from the output we have solved the problem.
There’s an important point here. If one thread gets the lock but never gives it back, your program will be stuck. In this case person P1 is taking infinite time for his meeting. What will happen?
We called this problem race condition! A race condition occurs when two or more threads access shared data simultaneously, and at least one thread modifies the data, leading to unpredictable results.

Lets modify the program like the following.

def first_person_with_lock(lock):
lock.acquire()
print("First person booked the room.")

time.sleep(1)
print("First person vacates the room.")
lock.release() # task finished lock released.

def second_person_with_lock_with_additional_info(lock):
while True:
if lock.acquire(blocking=False):
print("room is free!")
break
else:
print("meeting room isn't free yet.....")
time.sleep(0.2) # every 0.2 sec check the availability.

print("Second person booked the room.")
time.sleep(1)
print("Second person vacates the room.")
lock.release()

if __name__ == '__main__':
lock = threading.Lock()

thread1 = Thread(target=first_person_with_lock, args=(lock,))
thread2 = Thread(target=second_person_with_lock_with_additional_info, args=(lock,))

thread1.start()
thread2.start()

thread1.join()
thread2.join()

"""
Sample Output

First person booked the room.
meeting room isn't free yet.....
meeting room isn't free yet.....
meeting room isn't free yet.....
meeting room isn't free yet.....
meeting room isn't free yet.....
First person vacates the room.
room is free!
Second person booked the room.
Second person vacates the room.
"""

We have added some additional condition to check the resource/meeting room is accessed by someone or not. If meeting room/resource is free only then second person is able to access the meeting room. And that is how we can overcome the race condition. So In summary

  • A lock has two states: locked and unlocked.
  • When a thread acquires a lock, it changes the lock’s state to locked, preventing other threads from acquiring it.
  • Other threads trying to acquire the lock will be blocked until the first thread releases the lock (changes it back to unlocked).

Thread in Class

We can create thread in object oriented way as well.

import threading
from threading import Thread
import time


class MyThread(Thread):

def __init__(self, name, sleep_time=2):
super().__init__()
self.name = name
self.sleep_time = sleep_time

def run(self):
print(f"Thread {self.name} is running......")
time.sleep(self.sleep_time)
print(f"Thread {self.name} is exiting......")

if __name__ == '__main__':
thread1 = MyThread("Thread1", sleep_time=3)
thread2 = MyThread("Thread2", sleep_time=1)

thread1.start()
thread2.start()

thread1.join()
thread2.join()

"""
Sample Output

Thread Thread1 is running......
Thread Thread2 is running......
Thread Thread2 is exiting......
Thread Thread1 is exiting......
"""

In this example, we define a custom MyThread class that inherits from the built-in threading.Thread class. We define a constructor that takes a thread_name and sleep_time argument, which are used to identify the thread. We also define a run() method, which is the method that will be executed when the thread starts.
In the run() method, we print a message indicating that the thread is starting, and sleeping time to simulate some work being done, and then print another message indicating that the thread is ending.
Finally, we use the join() method to wait for the two threads to finish their work before exiting the main thread.

See you in the next article ASAP. Happy coding…..

Key references

Shariful Islam Sharif
Shariful Islam Sharif

Written by Shariful Islam Sharif

I am a software engineer specializing in Python, Django, and Elasticsearch, now expanding my skills into DevOps to focus on infrastructure and automation.