Create your Python Project Structure

Create your File and Folder Management in Python

Did you know you can create folders and files in Python? Well now you do!

Python’s os and pathlib libraries offer a range of functions to interact with the file system. We'll start by discussing basic concepts and then move on to practical examples, demonstrating how to create multiple folders and files programmatically.

Whether you’re tidying up data for a project, automating backups, or setting up a complex directory structure for a web app, Python has got your back.

Welcome to my article about making folders and files! My name is CyCoderX and in this article, we’ll dive into creating multiple folders and files using Python’s built-in libraries like os and pathlib.

Interested in more Python and Data Science content?

Feel free to explore my Lists on Medium 😊🐍📊

Creating Multiple Folders with os Module

The os module in Python is a versatile library for interacting with the operating system. It provides various methods to handle files and directories.

Let's create a basic script that creates multiple folders using os.makedirs().

# basic
import os # To interact with os

base_path = "E:/User/Example/Folder1"

# Make multiple files
for i in range(10):
    os.makedirs(base_path+"Day"+str(i))
    print(f"Directory {i} created successfully.")
print("Done")

• The code creates multiple directories (folders) within the specified base path.
• The loop runs 10 times, creating directories named "Day0", "Day1", …, "Day9" under the base_path.
• For each iteration, it uses os.makedirs() to create the directories.
• The print() statement inside the loop displays a success message for each directory created.
• Finally, it prints "Done" after all directories are created

Now let’s say we want to make multiple folders from the current directory we are and place them inside different locations.

Example: Creating Nested Directories

import os

# Define the directory structure
directories = [
    "project",
    "project/src",
    "project/tests",
    "project/docs"
]

# Create each directory
for directory in directories:
    os.makedirs(directory, exist_ok=True)
    print(f"Directory {directory} created successfully.")

• Defining directory structure: We list out the directories we want to create in a hierarchical structure.
• Creating directories: The os.makedirs() function iterates through each directory path and creates the directories. The exist_ok=True parameter allows the function to bypass any existing directories without raising an error.

Unlocking Python’s Hidden Powers: The Collections Library

Creating Multiple Files with pathlib Module

The pathlib module, introduced in Python 3.4, offers an object-oriented approach to handling filesystem paths. It's more intuitive and versatile compared to the traditional os module.

Example: Creating Multiple Files in Different Directories

from pathlib import Path

# Define the directory structure and file names
structure = {
    "project/src": ["main.py", "utils.py"],
    "project/tests": ["test_main.py", "test_utils.py"],
    "project/docs": ["readme.md", "changelog.md"]
}

# Create directories and files
for folder, files in structure.items():
    Path(folder).mkdir(parents=True, exist_ok=True)  # Create directories
    for file in files:
        file_path = Path(folder) / file 
        file_path.touch(exist_ok=True)
        print(f"File {file_path} created successfully.")

In this example:

• We define a dictionary where keys are directory paths and values are lists of file names to be created in those directories.
• The Path.mkdir() method creates directories, and the Path.touch() method creates empty files.

Explanation:

1. Importing pathlib module: We start by importing the Path class from the pathlib module.
2. Defining directory structure and file names: We use a dictionary to map directory paths to file names.
3. Creating directories and files:

• The Path.mkdir() method creates the specified directories, with parents=True ensuring that all parent directories are created as well.
• The Path.touch() method creates empty files. The exist_ok=True parameter prevents any error if the file already exists.
• the line file_path = Path(folder) / file creates a Path object representing the full path to a file within a specific folder. This path is then used to create or touch (update the access timestamp) the file.

This approach leverages the power of pathlib for a more readable and maintainable code structure.

Supercharge Your Python Code with Type Hinting

Combining Folder and File Creation with Error Handling

In real-world applications, robust error handling is crucial. And when we are talking about error handling usually what we usually mean is we need to add try and except clauses!

Let’s combine our previous examples and add error handling to ensure our script runs smoothly even if it encounters issues.

Example: Robust Directory and File Creation

import os
from pathlib import Path

# Define the directory structure and file names
structure = {
    "project/src": ["main.py", "utils.py"],
    "project/tests": ["test_main.py", "test_utils.py"],
    "project/docs": ["readme.md", "changelog.md"]
}

# Function to create directories and files with error handling
def create_structure(structure):
    for folder, files in structure.items():
        try:
            Path(folder).mkdir(parents=True, exist_ok=True)
            print(f"Directory {folder} created successfully.")
        except Exception as e:
            print(f"Error creating directory {folder}: {e}")
            continue
        
        for file in files:
            try:
                file_path = Path(folder) / file
                file_path.touch(exist_ok=True)
                print(f"File {file_path} created successfully.")
            except Exception as e:
                print(f"Error creating file {file_path}: {e}")

# Create the directory and file structure
create_structure(structure)

In this example:

• We define the directory structure and file names as before.
• We create a function create_structure() that includes error handling for both directory and file creation.

Explanation:

1. Combining os and pathlib: We use pathlib for a clean and Pythonic way to manage paths while retaining the familiar os methods for broader compatibility.
2. Error Handling: We wrap the directory and file creation logic in try-except blocks to catch and handle any exceptions, ensuring the script doesn’t crash unexpectedly.

By combining these techniques, we create a robust solution for managing files and directories in Python, capable of handling complex structures and potential errors gracefully.

Get CyCoderX’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Now, with these examples and explanations, you should now have a solid understanding of how to create multiple folders and files in Python, using both the os and pathlib modules.

Python Classes and Objects: An Essential Introduction

Master Python’s Enumerate for Efficient Coding

Practical Application — Project Initialization Script

Now that we’ve covered the basics of creating folders and files, let’s put it all together in a practical example. We’ll create a script to initialize a standard project structure. This can be incredibly useful when starting new projects, ensuring a consistent and organized setup.

Example: Project Initialization Script

Imagine you frequently start new projects with a similar structure. We can automate the creation of this structure with a Python script.

Project Structure

We’ll create the following structure:

my_new_project/
├── src/
│   ├── main.py
│   └── utils.py
├── tests/
│   ├── test_main.py
│   └── test_utils.py
└── docs/
    ├── readme.md
    └── changelog.md

Script to Create the Project Structure

import os
from pathlib import Path

# Define the directory structure and file names
project_structure = {
    "src": ["main.py", "utils.py"],
    "tests": ["test_main.py", "test_utils.py"],
    "docs": ["readme.md", "changelog.md"]
}

# Function to create directories and files
def create_project_structure(base_path, structure):
    base_path = Path(base_path)
    
    for folder, files in structure.items():
        folder_path = base_path / folder
        try:
            folder_path.mkdir(parents=True, exist_ok=True)
            print(f"Directory {folder_path} created successfully.")
        except Exception as e:
            print(f"Error creating directory {folder_path}: {e}")
            continue
        
        for file in files:
            file_path = folder_path / file
            try:
                file_path.touch(exist_ok=True)
                print(f"File {file_path} created successfully.")
            except Exception as e:
                print(f"Error creating file {file_path}: {e}")

# Specify the base path for the new project
base_path = "my_new_project"

# Create the project structure
create_project_structure(base_path, project_structure)

In this example:

• We define the project structure in a dictionary where keys are subdirectories and values are lists of filenames.
• The create_project_structure() function takes a base path and the structure dictionary, creating the necessary directories and files within the specified base path.

Explanation:

1. Defining the Project Structure: We outline the desired folder and file structure in a dictionary.
2. Creating the Structure:

• The script first ensures the base project directory exists.
• It then iterates through each folder and file, creating them as specified.

3. Error Handling: As in the previous example, we include error handling to manage any issues that arise during directory or file creation.

Running the Script

Save the script as init_project.py and run it from the command line:

python init_project.py

This will create the my_new_project directory with the specified subdirectories and files.

5 Things I Wish I Knew Earlier in Python! 🐍

Conclusion

In this article, we explored various methods to create multiple folders and files in Python using the os and pathlib modules. We started with simple examples and progressively built up to a comprehensive project initialization script. These techniques can streamline your workflow and ensure consistency across projects.

By leveraging Python’s file management capabilities, you can automate repetitive tasks, save time, and reduce the likelihood of errors. Whether you’re a seasoned developer or just starting, these tools will enhance your productivity and organizational skills.

Happy coding! If you have any questions or suggestions, feel free to leave a comment.

Displaying DataFrames Side By Side in Jupyter Notebook

Photo by Diana Polekhina on Unsplash

Final Words:

Thank you for taking the time to read my article.

This article was first published on medium by CyCoderX.

Hey There! I’m CyCoderX, a data engineer who loves crafting end-to-end solutions. I write articles about Python, SQL, AI, Data Engineering, lifestyle and more!

Join me as we explore the exciting world of tech, data and beyond!

For similar articles and updates, feel free to explore my Medium profile.

If you enjoyed this article, consider following for future updates.

Interested in Python content and tips? Click here to check out my list on Medium.

Interested in more SQL, Databases and Data Engineering content? Click here to find out more!

What did you think about this article? Let me know in the comments below … or above, depending on your device 🙃

In Plain English 🚀

Thank you for being a part of the In Plain English community! Before you go:

• Be sure to clap and follow the writer ️👏️️
• Follow us: X | LinkedIn | YouTube | Discord | Newsletter
• Visit our other platforms: CoFeed | Differ
• More content at PlainEnglish.io