Cameras and OpenCV

We've decided to put a camera on Dangly Too (hopefully two if we have time to upgrade to a Pi 5). But the camera does little than take a still or video without additional software. For this, one of the best tools to use is the open-source computer-vision package OpenCV. It has a massive wealth of analysis that can be used directly from Python, and it can also be very fast and easily handle 25 frame-per-second for some quite complex processing.

Installation

In theory, you can use "pip" to install the OpenCV library and its dependencies for use in Python. In practice this takes several hours rarely works! Instead, it is far easier to install as a "apt" package which is pretty fast and installs most of what you nee:

sudo apt install python3-opencv

Depending on what analysis you intend to use, you may need to install addional packages too, e.g.

sudo apt install libopenblas-dev

Note: As this is an "apt" package that installs Python libraries, it will NOT automatically add to existing Python virtual environments. You will need to either re-create your virtual environments or manually add symbolic links to the multiple libraries and .so's that get installed (too much info for this blog).

Legacy Camera vs. LibCamera Libraries

In the latest version of the Pi OS operating system, a new camera interface library has replaced the older "legacy" interface (raspistill etc) that's been there a long time.

Previously, you needed to enable the camera in the config (e.g. sudo raspi-config) and then you could use PiCamera or OpenCV to capture images with ease. Unfortunately newer cameras now can only be used with the latest libcamera interface, so we're going to stick with this. For this, you MUST NOT enable the legacy interface, but instead ensure you have the appropriate modules installed for your chosen camera. Luckily the standard cameras (and compatibles) are already installed.

Sample Code

If we want to capture an analyse a continuous stream of video in OpenCV, you can use the PiCamera2 python library as a replacement for the older PiCamera one.

pip install picamera2

Now we have this, we can capture and image stream and analyse. This looks for faces in the picture and draws a box around what it finds. Neat!

import cv2
from picamera2 import Picamera2

# Grab images as numpy arrays and leave everything else to OpenCV.
face_detector = cv2.CascadeClassifier("./haarcascade_frontalface_default.xml") #"/usr/share/opencv4/haarcascades/haarcascade_frontalface_default.xml")
cv2.startWindowThread()

picam2 = Picamera2()
picam2.configure(picam2.create_preview_configuration(main={"format": 'XRGB8888', "size": (640, 480)}))
picam2.start()

while True:
    im = picam2.capture_array()

    grey = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
    faces = face_detector.detectMultiScale(grey, 1.1, 5)

    for (x, y, w, h) in faces:
        cv2.rectangle(im, (x, y), (x + w, y + h), (0, 255, 0))

    cv2.imshow("Camera", im)
    cv2.waitKey(1)

You'll also need the classifier file haarcascade_frontalface_default.xml, which you can download from here:

https://github.com/opencv/opencv/blob/4.x/data/haarcascades/haarcascade_frontalface_default.xml