Real-Time Object Detection with Raspberry Pi AI HAT

Real-Time Object Detection with Raspberry Pi AI HAT

1. Overview

The Raspberry Pi 5 is a very useful and adaptable platform for various embedded and edge computing applications. However, when working with computer vision models, the performance on the single board computer can lack in either accuracy, speed, or both. To address this use case, the Raspberry Pi company released the Raspberry Pi AI Kit and Raspberry Pi AI HAT+ with dedicated neural network accelerators.

This additional hardware enables the Raspberry Pi 5 to run models like YOLO with live camera streams while keeping the computer's compact form factor.

What You'll Build

Prerequisites

The active cooler is recommended by Raspberry Pi for the best performance.

The Raspberry Pi AI Kit may still be for sale but it has been replaced by the AI HAT+ going forward, which is why it is recommended here. Either the 13 TOPS or 26 TOPS version will work in this codelab.

Follow the official documentation for instructions about installing the AI HAT+ on the Raspberry Pi 5.

What You'll Need

What You'll Learn

Watch the Video

See a demonstration of the AI HAT+ object detection:

Object Detection with Raspberry Pi AI HAT | DEMO

2. Set up your Raspberry Pi

The Raspberry Pi boots from a microSD card. You need to install Raspberry Pi OS on the microSD card that you will use with your Pi. For more details about alternative methods of setting up your Raspberry Pi, refer to the Viam docs.

Install Raspberry Pi OS

  1. Connect the microSD card to your computer.

  2. Download the Raspberry Pi Imager and launch it.

  3. Click CHOOSE DEVICE. Select your model of Pi, which is Raspberry Pi 5.

  4. Click CHOOSE OS. Select Raspberry Pi OS (64-bit) from the menu.

  5. Click CHOOSE STORAGE. From the list of devices, select the microSD card you intend to use in your Raspberry Pi.

  6. Configure your Raspberry Pi for remote access. Click Next. When prompted to apply OS customization settings, select EDIT SETTINGS.

  7. Check Set hostname and enter the name you would like to access the Pi by in that field, for example, test.

  8. Select the checkbox for Set username and password and set a username (for example, your first name) that you will use to log into the Pi. If you skip this step, the default username will be pi (not recommended for security reasons). And specify a password.

  9. Connect your Pi to Wi-Fi so that you can run viam-server wirelessly. Check Configure wireless LAN and enter your wireless network credentials. SSID (short for Service Set Identifier) is your Wi-Fi network name, and password is the network password. Change the section Wireless LAN country to where your router is currently being operated.

  10. Select the SERVICES tab, check Enable SSH, and select Use password authentication.

  11. Save your updates, and confirm YES to apply OS customization settings. Confirm YES to erase data on the USB flash drive. You may also be prompted by your operating system to enter an administrator password. After granting permissions to the Imager, it will begin writing and then verifying the Linux installation to the USB flash drive.

  12. Remove the USB flash drive from your computer when the installation is complete.

Connect with SSH

  1. Place the microSD card into your Raspberry Pi and boot the Pi by plugging it in to an outlet. A red LED will turn on to indicate that the Pi is connected to power.
  2. Once the Pi is started, connect to it with SSH. From a command line terminal window, enter the following command. The text in <> should be replaced (including the < and > symbols themselves) with the user and hostname you configured when you set up your Pi.
ssh <USERNAME>@<HOSTNAME>.local

  1. If you are prompted "Are you sure you want to continue connecting?", type "yes" and hit enter. Then, enter the password for your username. You should be greeted by a login message and a command prompt.

  2. Update your Raspberry Pi to ensure all the latest packages are installed

sudo apt update
sudo apt upgrade

With the OS installed, it's time to set up the system packages for the AI HAT+.

3. Set up the hardware

Connect to the Pi with SSH, if you're not still connected from the previous step.

ssh <USERNAME>@<HOSTNAME>.local

Enable PCIe Gen 3.0 speeds for the best performance

  1. Open the Raspberry Pi Configuration interface:
sudo raspi-config

  1. Select Advanced Options

  2. Select PCIe Speed

  3. Choose Yes to enable PCIe Gen 3 mode

  4. Select Finish to exit the configuration interface

  5. Reboot the device

sudo reboot
  1. Reconnect to the Pi with SSH to continue.

Set up system dependencies

  1. Install the hailo-all package, which contains the firmware, device drivers, and processing libraries for the AI HAT+:
sudo apt install -y hailo-all

This may take a few minutes depending on your network speed. 2. Reboot the device

sudo reboot
  1. Reconnect to the Pi with SSH
  2. Verify everything is running correctly:
hailortcli fw-control identify

You should see output similar to the following:

Executing on device: 0000:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.17.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8
Device Architecture: HAILO8L
Serial Number: HLDDLBB234500054
Part Number: HM21LB1C2LAE
Product Name: HAILO-8L AI ACC M.2 B+M KEY MODULE EXT TMP

Add your webcam

  1. Connect the webcam's USB lead to any USB port on your Pi. The webcam transmits video data to the Raspberry Pi through the USB interface.

Now that we have physically connected our hardware components, place them in a spot with a good view of the traffic. In the next section, we'll configure our machine.

4. Configure your machine and peripherals

Configure your machine

  1. In the Viam app under the LOCATIONS tab, create a machine by typing in a name and clicking Add machine.

  2. Click View setup instructions.

  3. To install viam-server on the Raspberry Pi device that you want to use, select the Linux / Aarch64 platform for the Raspberry Pi, and leave your installation method as viam-agent.

  4. Use the viam-agent to download and install viam-server on your Raspberry Pi. Follow the instructions to run the command provided in the setup instructions from the SSH prompt of your Raspberry Pi.

  5. The setup page will indicate when the machine is successfully connected.

Configure your webcam

  1. In the Viam app under the CONFIGURE tab, click the + icon in the left-hand menu and select Component.

  2. Select camera, and find the webcam module. This adds the module for working with a USB webcam. Leave the default name camera-1 for now.

  3. Notice adding this component adds the webcam hardware component called camera-1. From the Attributes section of the panel, select a video_path.

  4. Click Save in the top right. This may take a moment to apply your configuration changes.

  5. At the bottom of the camera-1 panel, expand the TEST section to ensure you have configured the camera properly and see a video feed.

Configure your vision service

  1. In the Viam app under the CONFIGURE tab, click the + icon in the left-hand menu and select Service.

  2. Select vision, and find the hailo-rt module. This adds the module for working with Hailo Runtime used by the AI HAT+. Select "Add module". Leave the default name vision-1 for now.

  3. Notice adding this service adds the computer vision software service called vision-1. From the Depends on section of the panel, select camera-1 from the "Search resources" dropdown.

  4. Click Save in the top right. This may take a moment to apply your configuration changes.

  5. At the bottom of the vision-1 panel, expand the TEST section to ensure you have configured the service properly and see images from camera-1 with object detection boxes on top.

Configure your detections sensor

  1. In the Viam app under the CONFIGURE tab, click the + icon in the left-hand menu and select Component.
  2. Select sensor, and find the detections module. This adds a module for capturing object detection data from a vision service. Leave the default name sensor-1 for now.
  3. Notice adding this component adds the sensor component called sensor-1. From the Attributes section of the panel, add the following JSON configuration.
{
     "camera": "camera-1",
     "detector": "vision-1",
     "labels": ["car", "bus", "person"]
}

  1. From the Data capture section of the panel, click "+ Add method". Set the "Method" as "Readings", "Frequency (hz)" as 1. Click "Create data management service" in the banner within the panel to automatically create a Data Management Service called data_manager-1.

  2. Click Save in the top right. This may take a moment to apply your configuration changes.

  3. At the bottom of the sensor-1 panel, expand the TEST section to ensure you have configured the sensor properly and see a list of the configured labels with the number of detections refreshed regularly.

With all the components and services in place, you can move on to creating a live tele-operations dashboard for your machine!

5. Create your traffic dashboard

This step walks through how to use the teleop (or tele-operations) feature of the Viam app.

Configure your teleop workspace

  1. In the Viam app, return to the home page.

  2. Then under the TELEOP tab, create a new dashboard by clicking Create workspace.

  3. Click on "untitled-workspace" to set the name of your dashboard.

  4. Click "Select machine" and choose your previously configured machine from the list.

Configure camera stream widget

  1. Click "+ Add widget" and select "Camera stream".

  2. Notice adding this widget creates a panel called Camera. Select camera-1 from the "Camera name" field, keep the "Refresh type" as "Live".

Configure time series graph widget

  1. Click "+ Add widget" and select "Time series".

  2. Notice adding this widget creates a new panel called Time series. Set the "Title" to "Traffic" and "Time range (min)" to 30.

  3. Under the Lines section of the panel, select sensor-1 for the "Resource name", Readings for "Capture method", cars for "Title", and readings.car for "Path".

  4. Add new lines by clicking "+ Add line". Create a line for each remaining label captured by the sensor: bus and person.

  5. Click "Save changes" to see the camera stream and data updating live.

6. Conclusion And Resources

What you learned

Building advanced scenarios with Viam and the Raspberry Pi AI HAT+

At this point, you have created an edge device that can perform real-time object detection and monitor it remotely from anywhere! You can keep building on this project with additional features:

The default YOLO-based models can do more than detect traffic! You can find the full list of identifiable objects included in the module repo: https://github.com/HipsterBrown/viam-pi-hailo-ml/

If none of those items suit your project requirements, you can explore the full suite of model types in the Hailo Model Zoo or retrain a model on a custom dataset.

In addition to the project ideas mentioned above, consider other ways to continue your journey with Viam.

Related Viam resources