# Use an AprilTag scanner ## 1. Overview **Why would you want to detect AprilTags?** [AprilTags](https://april.eecs.umich.edu/software/apriltag), similar to QR codes, can encode a variety of information and are widely used in robotics and computer vision applications. Unlike QR codes, AprilTags are designed for scenarios where precise localization, identification, and orientation are critical. They are commonly used for tasks such as guiding autonomous vehicles, tracking objects, and enhancing augmented reality systems. In this codelab, you'll learn how to detect and decode AprilTags using a Viam module. We'll leverage the [`apriltag`](https://pypi.org/project/apriltag/) and [OpenCV](https://pypi.org/project/opencv-python/) Python libraries to process images from a camera and extract information encoded in AprilTags. By the end, you'll have a working solution that can identify AprilTags in real-time, enabling customizable actions for a variety of applications, from navigation to interactive robotics. ## What You'll Build - A camera that can detect and decode AprilTags ## Prerequisites - A computer with MacOS, Windows, or Linux to flash your Raspberry Pi and configure the device's components using the Viam app - Hardware and supplies requirements - 1 - [Raspberry Pi 4](https://a.co/d/fc3JUMm) - 1 - microSD card to use with your Pi - 1 - power supply for your Pi - 1 - USB web camera ## What You'll Need - All the hardware components listed in prerequisites. - Sign up for a free Viam account, and then [sign in](https://app.viam.com/fleet/dashboard) to the Viam app ## What You'll Learn - How to use a webcam to scan and detect AprilTags - How to configure and test a device's components using Viam - How to use a vision service in Viam - How to use modules from the Viam registry ## Watch the Video See a demonstration and overview of the AprilTags scanner in this video. [AprilTag detector | DEMO](https://www.youtube.com/watch?v=7lknmKzZcn4) ## 2. Set up your Raspberry Pi The Raspberry Pi boots from a USB flash drive (or microSD card). You need to install Raspberry Pi OS on a USB flash drive that you will use with your Pi. For more details about alternative methods of setting up your Raspberry Pi, refer to the [Viam docs](https://docs.viam.com/installation/prepare/rpi-setup/#install-raspberry-pi-os). ### Install Raspberry Pi OS 01. Connect the USB flash drive (or microSD card) to your computer. 02. Download the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) and launch it. 03. Click **CHOOSE DEVICE**. Select your model of Pi, which is Raspberry Pi 4. 04. Click **CHOOSE OS**. Select **Raspberry Pi OS (64-bit)** from the menu. 05. Click **CHOOSE STORAGE**. From the list of devices, select the USB flash drive you intend to use in your Raspberry Pi. 06. Configure your Raspberry Pi for remote access. Click **Next**. When prompted to apply OS customization settings, select **EDIT SETTINGS**. 07. Check **Set hostname** and enter the name you would like to access the Pi by in that field, for example, `test`. 08. Select the checkbox for **Set username and password** and set a username (for example, your first name) that you will use to log into the Pi. If you skip this step, the default username will be `pi` (not recommended for security reasons). And specify a password. 09. Connect your Pi to Wi-Fi so that you can run `viam-server` wirelessly. Check **Configure wireless LAN** and enter your wireless network credentials. SSID (short for Service Set Identifier) is your Wi-Fi network name, and password is the network password. Change the section `Wireless LAN country` to where your router is currently being operated. 10. Select the **SERVICES** tab, check **Enable SSH**, and select **Use password authentication**. 11. **Save** your updates, and confirm `YES` to apply OS customization settings. Confirm `YES` to erase data on the USB flash drive. You may also be prompted by your operating system to enter an administrator password. After granting permissions to the Imager, it will begin writing and then verifying the Linux installation to the USB flash drive. 12. Remove the USB flash drive from your computer when the installation is complete. ### Connect with SSH 1. Place the USB flash drive into your Raspberry Pi and boot the Pi by plugging it in to an outlet. A red LED will turn on to indicate that the Pi is connected to power. 2. Once the Pi is started, connect to it with SSH. From a command line terminal window, enter the following command. The text in <> should be replaced (including the < and > symbols themselves) with the user and hostname you configured when you set up your Pi. ```bash ssh @.local ``` 3. If you are prompted "Are you sure you want to continue connecting?", type "yes" and hit enter. Then, enter the password for your username. You should be greeted by a login message and a command prompt. 4. Update your Raspberry Pi to ensure all the latest packages are installed ```bash sudo apt update sudo apt upgrade ``` ## 3. Configure your machine and peripherals ### Configure your machine 1. In [the Viam app](https://app.viam.com/fleet/dashboard) under the **LOCATIONS** tab, create a machine by typing in a name and clicking **Add machine**. 2. Click **View setup instructions**. 3. To install `viam-server` on the Raspberry Pi device that you want to use to communicate with and control your webcam, select the `Linux / Aarch64` platform for the Raspberry Pi, and leave your installation method as [`viam-agent`](https://docs.viam.com/how-tos/provision-setup/#install-viam-agent). 4. Use the `viam-agent` to download and install `viam-server` on your Raspberry Pi. Follow the instructions to run the command provided in the setup instructions from the SSH prompt of your Raspberry Pi. 5. The setup page will indicate when the machine is successfully connected. ### Add your USB webcam 1. Connect the USB webcam to the Raspberry Pi. 2. In [the Viam app](https://app.viam.com/fleet/locations), find the **CONFIGURE** tab. It's time to configure your hardware. 3. Click the **+** icon in the left-hand menu and select **Component**. 4. Select `camera`, and find the `webcam` module. This adds the module for working with a USB webcam. Leave the default name `camera-1` for now. 5. Notice adding this component adds the webcam hardware component called `camera-1`. The collapsible panel on the right corresponds to the part listed in the left sidebar. From the **Attributes** section of the panel, select a `video_path`. 6. Click **Save** in the top right to save and apply your configuration changes. 7. At the bottom of the `camera-1` panel, expand the **TEST** section to ensure you have configured the camera properly. ## 4. Add a vision service Now that your hardware is working the way you want it, it's time to add a vision service to detect and decode an AprilTag. 1. In the Viam app, click the **+** icon in the left-hand menu and select **Service**, and then `vision`. 2. Search for a module called `apriltag`. Then click **Add module**, and **Create** a new vision service called `vision-1`. 3. Notice this creates two new items in the left sidebar. The first is your new vision service called `vision-1`, and the second is your new `apriltag` module. 4. In the `vision-1` panel under the **Depends on** section, check the `camera-1` resource. This configures the vision service to depend on data coming in from the webcam. 5. **Save** your changes in the top right and wait a few moments for the configuration changes to take effect. 6. At the bottom of the `vision-1` panel, expand the **TEST** section to ensure you have configured the vision service properly. Point the webcam towards an AprilTag to see if the camera detects it. 7. If the camera detects an AprilTag, a bounding box will highlight the item in the video feed, decode the data, and display it on the right under **Labels**. In the example shown here, the data decoded was unique ID `234`. ## 5. Next Steps ### What you learned ### Building advanced features with Viam and AprilTags At this point, you have configured and tested your machine and webcam to detect and decode AprilTags, but nothing else is happening automatically, you can create [automatic processes](https://docs.viam.com/configure/processes/) that trigger actions when AprilTags are detected, similar to the [steps in this QR code detection codelab](https://codelabs.viam.com/guide/qrcode/index.html?index=..%2F..index#3). Now that you have a camera that detects and decodes AprilTags, you can: - [train and deploy a custom vision model](https://docs.viam.com/how-tos/train-deploy-ml/) to detect and classify other objects alongside AprilTags - extend the [apriltag module](https://app.viam.com/module/joyce/apriltag) in the Viam registry to use different tag families, add custom functionality such as pose detection, or trigger specific actions based on detected tags - [add more components and services](https://docs.viam.com/platform/#the-things-that-make-up-a-machine) to enhance the functionality of your machine, such as integrating advanced robotics controls or feedback systems ### Real-world applications and projects for AprilTags AprilTags are used in the physical world for tasks that require precise localization, identification, and orientation. They are used across various industries and projects, including: - **Navigation and Localization**: AprilTags serve as waypoints for autonomous navigation, enabling [roving robots](https://docs.viam.com/tutorials/configure/configure-rover/) to traverse offices, homes, or outdoor spaces with high accuracy. - **Access Control**: Secure access systems can link specific AprilTags to trigger actions like unlocking doors, disabling alarms, or raising barriers. - **Interactive Robotics**: Robots can use AprilTags to identify objects or trigger specific actions, such as picking up or delivering items to designated areas. - **Augmented Reality**: Physical objects tagged with AprilTags can be recognized and interacted with in augmented reality applications. - **Logistics and Warehousing**: AprilTags track and identify packages or inventory in warehouses and distribution centers, streamlining operations. - **Manufacturing**: They guide assembly line operations, ensuring precision in automated workflows. ### Notable real-world use cases - **New York City Subway**: AprilTags are being used [to improve accessibility for visually impaired and non-English speakers](https://www.timeout.com/newyork/news/qr-codes-will-make-navigating-the-subway-easier-for-the-visually-impaired-and-non-english-speakers-011724) by aiding navigation. - **FIRST Tech Challenge**: AprilTags [provide precise location and orientation for robots during competitions](https://ftc-docs.firstinspires.org/en/latest/apriltag/vision_portal/apriltag_intro/apriltag-intro.html), enabling autonomous navigation and task execution.