Use an AprilTag scanner

Use an AprilTag scanner

1. Overview

Why would you want to detect AprilTags? AprilTags, similar to QR codes, can encode a variety of information and are widely used in robotics and computer vision applications. Unlike QR codes, AprilTags are designed for scenarios where precise localization, identification, and orientation are critical. They are commonly used for tasks such as guiding autonomous vehicles, tracking objects, and enhancing augmented reality systems.

In this codelab, you'll learn how to detect and decode AprilTags using a Viam module. We'll leverage the apriltag and OpenCV Python libraries to process images from a camera and extract information encoded in AprilTags. By the end, you'll have a working solution that can identify AprilTags in real-time, enabling customizable actions for a variety of applications, from navigation to interactive robotics.

What You'll Build

Prerequisites

What You'll Need

What You'll Learn

Watch the Video

See a demonstration and overview of the AprilTags scanner in this video.

AprilTag detector | DEMO

2. Set up your Raspberry Pi

The Raspberry Pi boots from a USB flash drive (or microSD card). You need to install Raspberry Pi OS on a USB flash drive that you will use with your Pi. For more details about alternative methods of setting up your Raspberry Pi, refer to the Viam docs.

Install Raspberry Pi OS

  1. Connect the USB flash drive (or microSD card) to your computer.

  2. Download the Raspberry Pi Imager and launch it.

  3. Click CHOOSE DEVICE. Select your model of Pi, which is Raspberry Pi 4.

  4. Click CHOOSE OS. Select Raspberry Pi OS (64-bit) from the menu.

  5. Click CHOOSE STORAGE. From the list of devices, select the USB flash drive you intend to use in your Raspberry Pi.

  6. Configure your Raspberry Pi for remote access. Click Next. When prompted to apply OS customization settings, select EDIT SETTINGS.

  7. Check Set hostname and enter the name you would like to access the Pi by in that field, for example, test.

  8. Select the checkbox for Set username and password and set a username (for example, your first name) that you will use to log into the Pi. If you skip this step, the default username will be pi (not recommended for security reasons). And specify a password.

  9. Connect your Pi to Wi-Fi so that you can run viam-server wirelessly. Check Configure wireless LAN and enter your wireless network credentials. SSID (short for Service Set Identifier) is your Wi-Fi network name, and password is the network password. Change the section Wireless LAN country to where your router is currently being operated.

  10. Select the SERVICES tab, check Enable SSH, and select Use password authentication.

  11. Save your updates, and confirm YES to apply OS customization settings. Confirm YES to erase data on the USB flash drive. You may also be prompted by your operating system to enter an administrator password. After granting permissions to the Imager, it will begin writing and then verifying the Linux installation to the USB flash drive.

  12. Remove the USB flash drive from your computer when the installation is complete.

Connect with SSH

  1. Place the USB flash drive into your Raspberry Pi and boot the Pi by plugging it in to an outlet. A red LED will turn on to indicate that the Pi is connected to power.

  2. Once the Pi is started, connect to it with SSH. From a command line terminal window, enter the following command. The text in <> should be replaced (including the < and > symbols themselves) with the user and hostname you configured when you set up your Pi.

    ssh <USERNAME>@<HOSTNAME>.local

  3. If you are prompted "Are you sure you want to continue connecting?", type "yes" and hit enter. Then, enter the password for your username. You should be greeted by a login message and a command prompt.

  4. Update your Raspberry Pi to ensure all the latest packages are installed

   sudo apt update
   sudo apt upgrade

3. Configure your machine and peripherals

Configure your machine

  1. In the Viam app under the LOCATIONS tab, create a machine by typing in a name and clicking Add machine.

  2. Click View setup instructions.

  3. To install viam-server on the Raspberry Pi device that you want to use to communicate with and control your webcam, select the Linux / Aarch64 platform for the Raspberry Pi, and leave your installation method as viam-agent.

  4. Use the viam-agent to download and install viam-server on your Raspberry Pi. Follow the instructions to run the command provided in the setup instructions from the SSH prompt of your Raspberry Pi.

  5. The setup page will indicate when the machine is successfully connected.

Add your USB webcam

  1. Connect the USB webcam to the Raspberry Pi.

  2. In the Viam app, find the CONFIGURE tab. It's time to configure your hardware.

  3. Click the + icon in the left-hand menu and select Component.

  4. Select camera, and find the webcam module. This adds the module for working with a USB webcam. Leave the default name camera-1 for now.

  5. Notice adding this component adds the webcam hardware component called camera-1. The collapsible panel on the right corresponds to the part listed in the left sidebar. From the Attributes section of the panel, select a video_path.

  6. Click Save in the top right to save and apply your configuration changes.

  7. At the bottom of the camera-1 panel, expand the TEST section to ensure you have configured the camera properly.

4. Add a vision service

Now that your hardware is working the way you want it, it's time to add a vision service to detect and decode an AprilTag.

  1. In the Viam app, click the + icon in the left-hand menu and select Service, and then vision.

  2. Search for a module called apriltag. Then click Add module, and Create a new vision service called vision-1.

  3. Notice this creates two new items in the left sidebar. The first is your new vision service called vision-1, and the second is your new apriltag module.

  4. In the vision-1 panel under the Depends on section, check the camera-1 resource. This configures the vision service to depend on data coming in from the webcam.

  5. Save your changes in the top right and wait a few moments for the configuration changes to take effect.

  6. At the bottom of the vision-1 panel, expand the TEST section to ensure you have configured the vision service properly. Point the webcam towards an AprilTag to see if the camera detects it.

  7. If the camera detects an AprilTag, a bounding box will highlight the item in the video feed, decode the data, and display it on the right under Labels. In the example shown here, the data decoded was unique ID 234.

5. Next Steps

What you learned

Building advanced features with Viam and AprilTags

At this point, you have configured and tested your machine and webcam to detect and decode AprilTags, but nothing else is happening automatically, you can create automatic processes that trigger actions when AprilTags are detected, similar to the steps in this QR code detection codelab.

Now that you have a camera that detects and decodes AprilTags, you can:

Real-world applications and projects for AprilTags

AprilTags are used in the physical world for tasks that require precise localization, identification, and orientation. They are used across various industries and projects, including:

Notable real-world use cases