Welcome to SparkFun's MicroPython port of OpenCV! This is the first known MicroPython port of OpenCV, which opens up a whole new world of vision processing abilities on embedded devices in a Python environment!

As the first port, there may be incomplete or missing features, and some rough edges. For example, we have only implemented support for the Raspberry Pi RP2350 so far, and some of the build procedures are hard-coded for that. We'd be happy to work with the community to create an official port in the future, but until then, this repo is available and fully open-source for anyone to use!

Below are example code snippets of features avaiable in this port of OpenCV. We've done our best to make it as similar as possible to standard OpenCV, but there are some necessary API changes due to the limitations of MicroPython.

# Import OpenCV, just like any other Python environment!
import cv2 as cv

# Import ulab NumPy and initialize an image, almost like any other Python
# environment!
from ulab import numpy as np
img = np.zeros((240, 320, 3), dtype=np.uint8)

# Call OpenCV functions, just like standard OpenCV!
img = cv.putText(img, "Hello OpenCV!", (50, 200), cv.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)
img = cv.Canny(img, 100, 200)

# Call `cv.imshow()`, almost like standard OpenCV! Instead of passing a window
# name string, you pass a display driver that implements an `imshow()` method
# that takes a NumPy array as input.
cv.imshow(display, img)

# Call `cv.waitKey()`, just like standard OpenCV! Unlike standard OpenCV, this
# waits for a key press on the REPL instead of a window, and it is not necessary
# to call after `cv.imshow()` because display drivers show images immediately.
key = cv.waitKey(0)

# Use a camera, similar to standard OpenCV! `cv.VideoCapture()` is not used in
# MicroPython-OpenCV, because a separate camera driver that implements the same
# methods as the OpenCV `VideoCapture` class must be initialized separately.
camera.open()
success, frame = camera.read()
camera.release()

# Call `cv.imread()` and `cv.imwrite()` to read and write images to and from
# the MicroPython filesystem, just like standard OpenCV! It can also point to an
# SD card if one is mounted for extra storage space.
img = cv.imread("path/to/image.png")
success = cv.imwrite("path/to/image.png", img)

For full examples, see our Red Vision repo.

Limit your expectations. OpenCV typically runs on full desktop systems containing processors running at GHz speeds with dozens of cores optimized for computing speed and GB of RAM. In contrast, microcontrollers processors typically run at a few hundred MHz with 1 or 2 cores optimized for low power consumtion with a few MB of RAM. Exact performance depends on many things, including the processor, vision pipeline, image resolution, colorspaces used, RAM available, etc.

If you want best performance, keep in mind is that MicroPython uses a garbage collector for memory management. If images are repeatedly created in a vision pipeline, RAM will be consumed until the garbage collector runs. The collection process takes longer with more RAM, so this can result in noticable delays during collection (typically a few hundred milliseconds). To mitigate this, it's best to pre-allocate arrays and utilize the optional dst argument of OpenCV functions so memory consumption is minimized. Pre-allocation also helps improve performance, because allocating memory takes time.

Below are some typical execution times for various OpenCV functions. All were tested on a Raspberry Pi RP2350 with a 320x240 test image.

Below is a list of all OpenCV functions included in the MicroPython port of OpenCV. This section follows OpenCV's module structure.

Only the most useful OpenCV functions are included. The MicroPython environment is extremely limited, so many functions are omitted due to prohibitively high RAM and firmware size requirements. Other less useful functions have been omitted to reduce firmware size. If there are additional functions you'd like to be included, see #Contributing.

If you need help understanding how to use these functions, see the documentation link for each function. You can also check out OpenCV's Python Tutorials and other tutorials online for more educational experience. This repository is simply a port of OpenCV, so we do not document these functions or how to use them, except for deviations from standard OpenCV.

Standard OpenCV leverages the host operating system to access hardware, like creating windows and accessing cameras. MicroPython does not have that luxury, so instead, drivers must be implemented for these hardware devices. Take a look at our Red Vision repo for examples. This leads to necessary API changes for functions like cv.imshow().

As of writing, the OpenCV firmware adds over 3MiB on top of the standard MicroPython firmware, which itself be up to 1MiB in size (depending on platform and board). You'll also want some storage space, so a board with at least 8MB of flash is recommended.

PSRAM is basically a requirement to do anything useful with OpenCV. A single 320x240 RGB888 frame buffer requires 225KiB of RAM; most microcontrollers only have a few hundred KiB of SRAM. Several frame buffers can be needed for even simple vision pipelines, so you really need at least a few MiB of RAM available. The more the merrier!

Below are instructions to build the MicroPython-OpenCV firmware from scratch. Instructions are only provided for Linux systems.

1. Clone this repo and MicroPython

   • cd ~
     git clone https://github.com/sparkfun/micropython-opencv.git
     git clone https://github.com/micropython/micropython.git
     

2. Build the MicroPython cross-compiler

   • make -C micropython/mpy-cross -j4
     

3. Clone MicroPython submodules for your board

   • make -C micropython/ports/rp2 BOARD=SPARKFUN_XRP_CONTROLLER submodules
     

   • Replace rp2 and SPARKFUN_XRP_CONTROLLER with your platform and board name respectively
4. Set environment variables (if needed)
   • Some platforms require environment variables to be set. Example:

   • export PICO_SDK_PATH=~/micropython/lib/pico-sdk
     

5. Build OpenCV for your platform

   • make -C micropython-opencv PLATFORM=rp2350 --no-print-directory -j4
     

   • Replace rp2350 with your board's platform
6. Build MicroPython-OpenCV firmware for your board

   • export CMAKE_ARGS="-DSKIP_PICO_MALLOC=1 -DPICO_CXX_ENABLE_EXCEPTIONS=1" && make -C micropython/ports/rp2 BOARD=SPARKFUN_XRP_CONTROLLER USER_C_MODULES=~/micropython-opencv/micropython_opencv.cmake -j4
     

   • Replace rp2 and SPARKFUN_XRP_CONTROLLER with your platform and board name respectively
   • Replace the CMAKE_ARGS contents with whatever is required for your board's platform
   • Your firmware file(s) will be located in ~/micropython/ports/<port-name>/build-<board-name>/

Because OpenCV dramatically increases the firmware size, it may be necessary to define board variants that reduce the storage size to avoid it overlapping with the firmware. It is also beneficial to adjust the board name to include OpenCV or similar to help people identify that the MicroPython-OpenCV firmware is flashed to the board instead of standard MicroPython.

Below is the variant for the XRP Controller as an example. The variant is defined by creating a file called micropython/ports/rp2/boards/SPARKFUN_XRP_CONTROLLER/mpconfigvariant_RED_VISION.cmake with contents:

list(APPEND MICROPY_DEF_BOARD
    # Board name
    "MICROPY_HW_BOARD_NAME=\"SparkFun XRP Controller (Red Vision)\""
    # 8MB (8 * 1024 * 1024)
    "MICROPY_HW_FLASH_STORAGE_BYTES=8388608"
)

Some board definitions do not have #ifndef wrappers in mpconfigboard.h for MICROPY_HW_BOARD_NAME and MICROPY_HW_FLASH_STORAGE_BYTES. They should be added if needed so the variant can build properly.

Then, the firmware can be built by adding BOARD_VARIANT=<variant-name> to the make command when building the MicroPython-OpenCV firmware.

Only support for the Raspberry Pi RP2350 has been figured out, so the all requirements for adding new platforms is not fully known yet. However, it should be along the lines of:

1. Create a valid toolchain file for the platform
   • See rp2350.toolchain.cmake for reference
   • This loosely follow's OpenCV's platform definitions
2. Build OpenCV with the new platform

   • make -C micropython-opencv/opencv PLATFORM=<new-platform> --no-print-directory -j4
     

3. Create a new board for the new platform
   • See #Adding New Boards
4. Build MicroPython-OpenCV firmware for the new board

   • make -C micropython/ports/rp2 BOARD=<board-name> USER_C_MODULES=micropython-opencv/micropython_opencv.cmake -j4
     

Found a bug? Is there a discrepancy between standard OpenCV and MicroPython-OpenCV? Have a feature request?

First, please see if there is an existing issue. If not, then please open a new issue so we can discuss the topic!

Pull requests are welcome! Please keep the scope of your pull request focused (make separate ones if needed), and keep file changes limited to the scope of your pull request.