Working With Python Os Module

Photo by Markus Spiske on Unsplash

The Python Os module is highly utilized in Python for handling various low-level operating system tasks. It manages a variety of low-level operating system operations and aims to provide a uniform interface for different operating systems. This uniformity is particularly valuable when your application needs to run on both Windows and Unix-based systems. However, it’s worth noting that the os module also provides specific functionalities that may not be cross-platform.

Let’s explore some examples. You can run these code examples directly in Visual Studio Code. If you like, you can also use IPython, which is a user-friendly command-line tool for Python. You’ll find installation instructions, user guides, and settings for IPython here.

Some Python Os methods:

import os

# lists the contents of a directory
os.listdir(".") 

# renames a file or directory
os.rename("old_file", "new_file") 

# changes the permission settings of a file or directory
os.chmod("filename | directory", 0o777) 

# make a new directory
os.mkdir("directory name") 

# recursively create a directory path
os. makedirs("my_folder/my_subfolder") 

# remove or delete a file
os.remove("filename") 

# delete a single directory
os.rmdir("directory_path | directory name") 

# delete a tree of directories
os.removedirs("<directory_path>") 

# get stats about a file or directory.
os.stat("<filname | directory>") 

Working With Files and Folders Using the “os.path” Module

In Python, paths, whether binary or otherwise, can be represented using strings. The os.path module provides a wide range of methods for handling paths as strings. The os module aims to ensure cross-platform compatibility, and its os.path sub-module is no exception to this rule. It adapts path interpretation to the current operating system, using forward slashes ( / ), for directory separation on Unix-like systems and backward slashes ( \ ) on Windows. This enables your program to dynamically create paths that are compatible with the current system.

One of the most frequently used features of os.path is the ability to split and join paths. There are three methods commonly used for this purpose — split, basename, and dirname.

Here is a sample:

import os

# get the current working directory
current_directory = os.getcwd()

Remember to import the Os module in your code to access these functionalities.

Using split to separate the leaf level from the parent path:

import os

# split the leaf level of the path from the parent path
os.path.split(current_directory)

Using dirname to get the parent path:

import os

# returns the parent path
os.path.dirname(current_directory)

Using the basename to get the leaf name | file name:

import os

# returns the leaf name
os.path.basename(current_directory)

The os.path.dirname function in Python’s Os module allows you to walk up a directory tree. This means you can use it to move from a lower-level directory to its parent or any higher-level directory within the directory structure.

Let’s do this using a “while” loop:

import os

current_directory = os.getcwd()
 
while os.path.basename(current_directory):
   current_directory = os.path.dirname(current_directory)
   print(current_directory)

What does this code do:

Firstly, we import the Os module. This module provides functions for interacting with the operating system, including file and directory operations. Secondly, the current_directory = os.getcwd() gets the current working directory using os.getcwd(). It assigns the path of the current directory to the variable current_directory.

The while loop continues as long as the os.path.basename(current_directory) is not an empty string. The condition os.path.basename(current_directory) checks whether the current directory has a parent directory. In other words, it continues until it reaches the root directory where os.path.basename(current_directory) would be an empty string.

For each iteration of the loop, the code uses os.path.dirname(current_directory) to move up one level in the directory hierarchy. It assigns the parent directory’s path to the current_directory variable. After moving up a level, the code prints the current directory’s path to the console.

With a “for” loop, we can do the same:

import os

current_directory = os.getcwd()
directory = []

for _ in range(current_directory.count(os.sep)):
   current_directory = os.path.dirname(current_directory)
   directory.insert(0, current_directory)

for dirs in reversed(directory):
   print(dirs)

Here, we import the os module and obtain the current working directory with the os.getcwd(), which is stored in the variable current_directory. An empty list: directory = […], is created to store the directory hierarchy. The for loop, iterates based on the number of directory separators (os.sep) in the current_directory. The purpose is to go through the directory tree, starting from the current directory's immediate parent and moving upwards.

On Unix-based systems (including Linux and macOS), os.sep is a forward slash (/). On Windows, os.sep is a backslash (\).

current_directory.count(os.sep) calculates how many times os.sep (the directory separator for the current operating system) appears in the current_directory path. This number corresponds to the depth of the directory in the hierarchy.

Get Victor Ehikioya Raeva’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

current_directory = os.path.dirname(current_directory) is used to get the parent directory's path in each iteration. directory.insert(0, current_directory) adds the parent directory’s path to the directory at the beginning. Maintaining the order from parent to child.

The second for loop iterates through the directory list in reverse after gathering the directory hierarchy. “For dirs in reversed(directory)” loops through each directory path in the directory list, starting from the root and moving down. print(dirs) prints each directory path to the terminal.

Searching Files with the Os Module

We will utilize the OS methods for file searching. In this case, our objective is to find a specific file, which could potentially be a configuration file, and we’ll name it “tree.py.” It’s essential to note that Python does not inherently expand paths or environment variables, as is common in Unix-like systems. Therefore, we need to handle this task explicitly before we can create the path for searching our target file.

Let’s start by defining a function called find_file( ). This function’s initial step is to examine the current working directory:

import os 

# case
def find_file(file_name="tree.py"):
    current_directory = os.getcwd()
    full_path = os.path.join(current_directory, file_name)

    print(f"Checking {full_path}")

    if os.path.exists(full_path):
        return full_path

In the find_file( ) function, we will proceed by extending the search to the user’s home directory. It’s important to note that we utilize the expanduser( ) function to expand the path, as previously mentioned:

    home_dir = os.path.expanduser("~/") # expanduser path to widen search!
    full_path = os.path.join(home_dir, file_name)

    print(f"Checking {full_path}")

    if os.path.exists(full_path):
        return full_path

Lastly, we expand on the “file” variable. The “file” variable is populated with a path relative to the current working directory, not an absolute or full path:

    file_path = os.path.abspath(__file__)
    parent_path = os.path.dirname(file_path)
    full_path = os.path.join(parent_path, file_name)

    print(f"Checking {full_path}")

    if os.path.exists(full_path):
        return full_path

Getting the result:

    print(f"File {file_name} has not been found")


# trigger
file_name = find_file()
if file_name:
    print(f"Found file at: {file_name}")

Putting it together:

import os

# case
def find_file(file_name="tree.py"):

    current_directory = os.getcwd()
    full_path = os.path.join(current_directory, file_name)
    print(f"Checking {full_path}")

    if os.path.exists(full_path):
        return full_path

    home_dir = os.path.expanduser("~/")
    full_path = os.path.join(home_dir, file_name)
    print(f"Checking {full_path}")

    if os.path.exists(full_path):
        return full_path

    file_path = os.path.abspath(__file__)
    parent_path = os.path.dirname(file_path)
    full_path = os.path.join(parent_path, file_name)
    print(f"Checking {full_path}")

    if os.path.exists(full_path):
        return full_path

    print(f"File {file_name} has not been found")


# trigger
file_name = find_file()
if file_name:
    print(f"Found file at: {file_name}")

This code defines a Python function called find_file, which is responsible for searching for a file with the name "tree.py" in different locations. The function accepts an optional parameter, file_name, which defaults to "tree.py." It starts by obtaining the current working directory using the os.getcwd( ) function and combines it with the file_name to create a full path. A message is printed to indicate that this path is being checked.

Next, it uses os.path.exists( ) to check if the file specified by full_path exists. If the file is found, the full path is returned. This process is repeated for two additional locations: the user's home directory and the directory where the script (this code) is located. If the file is found in any of these locations, the full path is returned. If the file is not found in any of these locations, a message is printed to indicate that the file has not been found. Finally, if file_name is not empty, a message is printed to show the location of the file.

However, this code is repetitive. If you look closely, you will notice that we used full_path multiple times. And we repeatedly check os.path.exists( ). We can refine and optimize this code to make it simpler and readable.

import os

# case
def find_file(file_name="tree.py"):
    list_dir = [
        os.getcwd(), 
        os.path.expanduser("~/"),  
        os.path.dirname(os.path.abspath(__file__))
    ]

    for path in list_dir:
        if path is not None:
            file_path = os.path.join(path, file_name)
            if os.path.exists(file_path):
                print(f"Checking {file_path}")
                return file_path

    print(f"File {file_name} has not been found")

# trigger
file_name = find_file()
if file_name:
    print(f"Found file at: {file_name}")

What this does this code do:

In this function, we perform a similar task as the previous one, but we’ve introduced a list inside the function for easier iteration and adjustment. A for loop goes through the list to create a file_path. If the file_path exists, it prints the full path of the file; otherwise, it indicates that the file was not found.

Interacting With the Operating System Through the “Os” Module

The Os module provides a range of attributes and functions to handle the operating system. Below are some case studies. You can execute these code examples either in Visual Studio Code or through the IPython shell.

import os

# case

# get the current working directory
os.getcwd()

# change the current working directory
os.chdir("<dir_name>")

os.getcwd()

# the os.environ holds the environment variable
# we set when the os module is loaded
os.environ.get("CONFIG")

# this is the setting and environment variable. 
# this setting exists for subprocesses' spawned 
# from the code
os.environ["CONFIG"] = "DEBUG"

os.environ.get("CONFIG")

# this is the login of the user in
# the terminal that spawned this process.
os.getlogin()

The primary use of the os module often involves retrieving configurations from environment variables. These configurations might include log levels or sensitive data like API keys.

Conclusion:

The Python Os module is a versatile tool for interacting with the operating system. It offers extensive functionality for file and directory operations, process management, environment variables, and system-specific tasks. Pros include platform independence, robust file handling, process control, and path management. However, its complexity and platform variability can be challenging. In DevOps, the “Os” module plays a vital role in automation, deployment, system control, and environment management, making it a key component for managing and configuring various aspects of the system and environment in a DevOps workflow.