Pybind11 Tutorial: Binding C++ Code to Python

C++ is known for its performance and system-level programming capabilities, while Python excels in ease of use and rapid development. Combining these two languages allows developers to leverage the strengths of both. Have you ever wondered how to create C++ code that runs from Python? If so, then this tutorial is for you.

This tutorial provides a comprehensive step-by-step guide to using Pybind11, a library designed to simplify the process of creating Python bindings for C++ code. It is lightweight, header-only, and supports seamless interoperability between Python and C++. By the end, you will be able to expose your C++ code to Python, enabling you to write high-performance code in C++ while maintaining Python’s simplicity for scripting and testing.

The index of this tutorial is:

1. Installing a C++ Compiler.
2. Write C++ Code.
3. Create a setup.py Script.
4. Build the Python Module.
5. Use the Created Python Module.
6. Advanced Topics (Class and NumPy).
7. Conclusion.

Let’s get started!

Installing a C++ Compiler

A compatible C++ compiler is essential for building Pybind11 modules. Here are instructions for common operating systems:

For Linux, install GCC (GNU Compiler Collection). Run these commands to install the GNU C++ compiler and related development tools.

sudo apt update
sudo apt install build-essential g++

Write C++ Code

The first step is to write the C++ code that you want to expose to Python. For this tutorial, we’ll implement simple arithmetic functions as an example.

C++ Example Functions

Write the following C++ functions in a file named (for example) example.cpp:

int add(int a, int b) {
    return a + b;
}

int subtract(int a, int b) {
    return a - b;
}

These functions perform basic arithmetic operations and will be exposed to Python later. This is a foundational step to ensure you have a clear understanding of how C++ functions are structured and implemented. Once this code is written, we can move forward with binding it to Python.

Create the PYBIND11_MODULE Function

The PYBIND11_MODULE function is where the magic happens. This function binds your C++ functions to Python, allowing Python code to call them as if they were native Python functions.

Add the following code to the same file (example.cpp) to define the Python module and bind the C++ functions. It can be created in a separate file but we are going to use a single file for simplicity.

#include <pybind11/pybind11.h>

PYBIND11_MODULE(example, m) {
    m.doc() = "pybind11 example module"; // Optional module docstring

    m.def("add", &add, "A function that adds two numbers",
          pybind11::arg("a"), pybind11::arg("b"));

    m.def("subtract", &subtract, "A function that subtracts two numbers",
          pybind11::arg("a"), pybind11::arg("b"));
}

The PYBIND11_MODULE macro defines the module given the name example. To bind individual functions, the def() method is used. Each bound function is given a Python-accessible name, a reference to the C++ implementation, and optional docstrings for better clarity in Python. The names of the arguments are specified using pybind11::arg() to make sure the function is well-documented.

Complete Code

Here is the complete code of the example.cpp file that includes both the pure C++ code and the Pybind11 code. This file integrates both the arithmetic functions and their bindings in one cohesive block. It represents the starting point for creating a functional Python module from C++.

#include <pybind11/pybind11.h>

int add(int a, int b) {
    return a + b;
}

int subtract(int a, int b) {
    return a - b;
}

PYBIND11_MODULE(example, m) {
    m.doc() = "pybind11 example module"; // Optional module docstring

    m.def("add", &add, "A function that adds two numbers",
          pybind11::arg("a"), pybind11::arg("b"));

    m.def("subtract", &subtract, "A function that subtracts two numbers",
          pybind11::arg("a"), pybind11::arg("b"));
}

Create a setup.py Script

After writing the C++ code, the next step is to set up a build system. Even though we can use CMake, we are going to create a setup.py script instead. This is because Python developers are more familiar with this script and easier to manage in the Python projects.

This is an example of the setup.py file.

from setuptools import setup, Extension
import pybind11

ext_modules = [
    Extension(
        "example",  # Module name
        ["example.cpp"],  # Source files
        include_dirs=[pybind11.get_include()],  # Include pybind11 headers
        language="c++",  # Specify C++ as the language
    )
]

setup(
    name="example",
    version="0.1",
    ext_modules=ext_modules,
)

The Extension class defines a Python extension module which is given the name example. A list of the source files are included to reference the C++ code. We only have the single file called example.cpp.

Using pybind11.get_include(), we ensure that the Pybind11 headers are included during compilation. Without this, the compiler won't find the required Pybind11 definitions.

Get Ahmed Gad’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Remember to specify the language of the source code (language="c++") to ensure compatibility with the C++ compiler.

Build the Python Module

Once the setup.py script is ready, building the module is straightforward. Just run the following command in the terminal to build the Python module.

python setup.py build

This will compile the C++ code and generate a Python module (e.g., example.cpython-<version>-<platform>.so) in the build directory. This step ensures that all dependencies and configurations are correctly compiled into a shared object file.

To make the module accessible system-wide or in your virtual environment, you can install the module using this command:

python setup.py install

This will install the module in your Python environment, allowing you to import it from anywhere.

Use the Created Python Module

Once the Python module is exported, then write a simple Python script (e.g. example.py) with this code. It imports the example module and call the add() and subtract() functions inside it.

Just make sure the .so file is in the same directory of the example.py script (unless you opted to install the module).

Run the code and check if things are working well.

import example

print(example.add(3, 5))       # Outputs: 8
print(example.subtract(10, 4)) # Outputs: 6

If the outputs match the expected values, your module is working correctly.

It might be tricky to get things working from the first time. This is why testing is a critical step to ensure that the bindings work as expected.

Reaching this section means you are done with using C++ code from Python. Congratulations! The remainder of this tutorial just highlights some other topics.

Advanced Topics (Class and NumPy)

Binding Classes

Pybind11 allows you to bind C++ classes, enabling object-oriented programming features in Python. The following example creates a simple class called Calculator. Its constructor accepts a single integer attribute called value. It also has a single method called add() which accepts an integer which is added to the value attribute.

#include <pybind11/pybind11.h>
​
class Calculator {
public:
    Calculator(int value) : value(value) {}
​
    int add(int x) { return value + x; }
​
private:
    int value;
};
​
PYBIND11_MODULE(example, m) {
    pybind11::class_<Calculator>(m, "Calculator")
        .def(pybind11::init<int>())
        .def("add", &Calculator::add);
}

The Pybind11 code binds the C++ class Calculator to Python, exposing its constructor and a member function. pybind11::class_ is used to bind the C++ class.

Follow the previous steps to build the Python module and try it.

Using NumPy

Pybind11 also provides support for integrating with NumPy, a powerful library for numerical computing in Python. This is particularly useful for applications that require manipulating large arrays of data efficiently.

The following code accepts a NumPy array and a scalar. Then the array elements are scaled by this scalar.

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
​
pybind11::array_t<double> scale_array(pybind11::array_t<double> input_array, double scalar) {
    // Request buffer information from the input NumPy array
    auto buf = input_array.request();
    
    // Check if the array is one-dimensional
    if (buf.ndim != 1) throw std::runtime_error("Input array must be 1-dimensional");
​
    // Access the raw data pointer
    double *ptr = static_cast<double *>(buf.ptr);
​
    // Create a new NumPy array to store the scaled values
    auto result = pybind11::array_t<double>(buf.size);
    auto result_buf = result.request();
    double *result_ptr = static_cast<double *>(result_buf.ptr);
​
    // Perform the scaling operation element-wise
    for (size_t i = 0; i < buf.size; ++i) {
        result_ptr[i] = ptr[i] * scalar;
    }
​
    return result; // Return the new NumPy array
}
​
PYBIND11_MODULE(example, m) {
    m.def("scale_array", &scale_array, "Scale a NumPy array by a scalar",
          pybind11::arg("input_array"), pybind11::arg("scalar"));
}

The request() method fetches metadata about the input NumPy array, such as dimensions, size, and a pointer to its data. The function ensures the input array is one-dimensional, throwing an error otherwise. This is just to make it simple.

The raw data pointer (buf.ptr) is cast to a double*, enabling direct access to array elements. Then a new NumPy array (result) is initialized to store the scaled values. Its buffer is also accessed for direct manipulation.

The for loop iterates over all elements of the input array, scales each by the given scalar, and writes the result to the new array. The function returns the new NumPy array to Python, ready for further use or analysis.

To test this function, use the following Python script:

import numpy as np
import example
​
# Create a NumPy array
array = np.array([1.0, 2.0, 3.0])
​
# Call the C++ function
scaled = example.scale_array(array, 2.0)
print(scaled)  # Outputs: [2. 4. 6.]

Conclusion

By following this tutorial, you have learned how to:

1. Write and structure your C++ code for Python bindings.
2. Use Pybind11 and setup.py to create and build a Python module from C++.
3. Use the resulting module in Python.
4. Work with advanced topics like class bindings and NumPy integration.

Pybind11 simplifies bridging the gap between Python and C++, enabling developers to build high-performance, Python-friendly extensions. With this knowledge, you can effectively combine the power of C++ with the simplicity of Python, unlocking new possibilities in your projects. Happy coding!