# People detection alert with computer vision ## 1. Overview This codelab is part of the [Viam for educators](https://education.viam.com/) program and provides a suggested delivery plan and resources for instructors interested in delivering a hands-on workshop. Take these resources, and make it your own! ## Workshop Overview - **Goal**: Introduce sensors and actuators as concepts and build a people detection alert - **Audience**: Beginner developers with basic programming and hardware skills - **Duration**: ~2-3 hours ## What You'll Teach - How to build an alert system with Computer Vision that triggers a buzzer when a person is detected ## 2. What participants will need Review the list of required components (hardware, software), and determine which materials are provided or required for participants to bring on their own. Provide instructions to participants, giving them ample time to prepare and complete prerequisites. ### Hardware and supplies requirements - 1 - [Raspberry Pi 4](https://a.co/d/fc3JUMm) - 1 - [USB flash drive](https://a.co/d/b0L2QL2) or [microSD card](https://a.co/d/42xzIiT) to use with the Pi, along with an adapter or reader - 1 - power supply for your Pi - 1 - USB web camera and a cable to connect it to your Pi - 1 - [passive piezo buzzer](https://a.co/d/34QRLLo) - 2 - jumper wires to connect the buzzer to the Pi - 1 - [optional] store-bought or 3D-printed Raspberry Pi case ( [like this one here](https://makerworld.com/en/models/62316#profileId-226178)) that provides access to GPIO pins - 1 - [optional] a breadboard can be used for your piezo buzzer, especially if you plan to wire additional components ### Software and other device requirements - A computer with MacOS, Windows, or Linux to flash your Raspberry Pi and configure the device's components, along with required software: - Preferred IDE, such as [VS Code](https://code.visualstudio.com/download) - [Python3](https://www.python.org/downloads/) - [Raspberry Pi Imager](https://www.raspberrypi.com/software/) - Sign up for a free Viam account, and then [sign in](https://app.viam.com/fleet/locations/) to the Viam app ## 3. Pre-workshop setup for instructors ### Learning Objectives Review the suggested learning objectives and adjust it according to your goals and audience. - How to use modules from [the Viam registry](https://docs.viam.com/registry/) - How to set up a control logic module on a Viam machine - How to use environment variables with your Viam machine - How to move control code to your machine ### Agenda Review the suggested agenda and adjust it according to your goals and audience. - **Introduction** (10 mins) - **Hardware Assembly** (30 mins) - **Software Setup** (30 mins) - **Hands-On Experiment** (40 mins) - **Q&A/Wrap-Up** (10 mins) ### Pre-workshop checklist ✅ Review these setup steps as you're planning the workshop to help prepare for it. 1. **Planning and Logistics** - **Verify equipment inventory**: Decide which materials are provided or required for participants to bring on their own. Order any materials and supplies that are provided to participants. - **Confirm workstation setup**: Verify the workshop room layout supports hardware work, such as tables for wiring the components. Ensure participants have power outlets and internet access. Have spare supplies, such as Raspberry Pis, adapters/readers, and cables, in case of failures. For virtual workshops, confirm logistics for breakout sessions, providing support, and checkpoints. - **Communicate prerequisites**: Share a participant guide prior to the workshop with learning objectives and instructions for installing software and preparing laptops. Provide a checklist for participants to confirm readiness. - **Establish support channels**: Determine how participants can get support for real-time troubleshooting (e.g. Slack or Discord channel for virtual workshops, additional helpers in the room for on-site workshops). - **Request Viam stickers**: Don't forget [to request Viam stickers](https://forms.gle/TFXBm7L9n8U4hLRj9) for the workshop. 2. **Content Preparation** - **Prepare workshop teaching materials**: Review the Delivery Plan, sample slide deck, and provided resources. Tailor these materials to suit your participants' needs. Decide whether to present them to the group or provide them for independent reference, such as through a web browser or printed handouts. - **Determine extensibility**: During the Hands-on Experiment portion of the delivery plan, there are many options to guide the workshop on a different learning path. Determine if this section will be self-guided exploration, a group exercise, or self-paced learning for participants to explore on their own after the workshop. - **Prepare code and documentation**: Review the provided code and add comments as necessary. - **Review discussion topics**: Review the suggested group discussion topics and adjust for your participants. - **Review quiz questions**: Review the suggested quiz questions and adjust for your participants. Determine if knowledge assessment will be completed throughout the workshop or afterwards. 3. **Dry Run and Testing** - **Test the workshop project**: Run through the build following the prepared documentation to ensure it works as expected. Simulate common issues and prepare troubleshooting tips for these scenarios. If physical space is limited, consider sharing a Raspberry Pi. - **Bring backups**: Final check to bring the backup supplies mentioned earlier during planning. ## 4. Delivery Plan - overview Feel free to make a copy of [this sample slide deck](https://docs.google.com/presentation/d/1ESOuSTW7XsjLQ2-n33JzsUHvDatJiWdTehb6pLKP7D0/edit?usp=sharing), customize it, and make it your own. During the workshop, instructors can present this customizable slide deck (see above) tailored to your specific workshop needs. Alternatively, learners can follow step-by-step instructions and checkpoints independently by referencing the pages linked below in a web browser. 1. [**Hardware Assembly**](https://codelabs.viam.com/guide/workshop-people/index.html#4) - Set up your Raspberry Pi - Connect with SSH - Set up your machine peripherals 2. [**Software Setup**](https://codelabs.viam.com/guide/workshop-people/index.html#5) - Configure your machine - Configure your Raspberry Pi board - Configure your webcam - Configure your ML model service - Configure your vision service - Configure your piezo buzzer 3. [**Hands-On Experiment**](https://codelabs.viam.com/guide/workshop-people/index.html#6) - Program your people detection alert - Configure and run a Viam control module - Finishing touches ## 5. Delivery Plan - Hardware Assembly ### Set up your Raspberry Pi The Raspberry Pi boots from a USB flash drive (or microSD card). You need to install Raspberry Pi OS on a USB flash drive that you will use with your Pi. For more details about alternative methods of setting up your Raspberry Pi, refer to the [Viam docs](https://docs.viam.com/installation/prepare/rpi-setup/#install-raspberry-pi-os). ### Install Raspberry Pi OS 01. Connect the USB flash drive (or microSD card) to your computer. 02. Download the [Raspberry Pi Imager](https://www.raspberrypi.com/software/) and launch it. 03. Click **CHOOSE DEVICE**. Select your model of Pi, which is Raspberry Pi 4. 04. Click **CHOOSE OS**. Select **Raspberry Pi OS (64-bit)** from the menu. 05. Click **CHOOSE STORAGE**. From the list of devices, select the USB flash drive you intend to use in your Raspberry Pi. 06. Configure your Raspberry Pi for remote access. Click **Next**. When prompted to apply OS customization settings, select **EDIT SETTINGS**. 07. Check **Set hostname** and enter the name you would like to access the Pi by in that field, for example `test` shown here. Each Pi should have a unique `hostname` to avoid conflicts on the shared local network, such as `-detection` or `-detection` if working in groups. 08. Select the checkbox for **Set username and password** and set a username (for example, your first name) that you will use to log into the Pi. If you skip this step, the default username will be `pi` (not recommended for security reasons). And specify a password. 09. Connect your Pi to Wi-Fi so that you can run `viam-server` wirelessly. Check **Configure wireless LAN** and enter your wireless network credentials. SSID (short for Service Set Identifier) is your Wi-Fi network name, and password is the network password. Change the section `Wireless LAN country` to where your router is currently being operated. 10. Select the **SERVICES** tab, check **Enable SSH**, and select **Use password authentication**. 11. **Save** your updates, and confirm `YES` to apply OS customization settings. Confirm `YES` to erase data on the USB flash drive. You may also be prompted by your operating system to enter an administrator password. After granting permissions to the Imager, it will begin writing and then verifying the Linux installation to the USB flash drive. 12. Remove the USB flash drive from your computer when the installation is complete. ### Connect with SSH 1. Place the USB flash drive into a USB port on your Raspberry Pi and boot the Pi by plugging it into an outlet. A red LED will turn on to indicate that the Pi is connected to power. 2. Once the Pi is started, connect to it with SSH. From a command line terminal window, enter the following command. The text in `< >` should be replaced (including the `<` and `>` symbols themselves) with the user and hostname you configured when you set up your Pi. ```bash ssh @.local ``` 3. If you are prompted "Are you sure you want to continue connecting?", type "yes" and hit enter. Then, enter the password for your username. You should be greeted by a login message and a command prompt. 4. Update your Raspberry Pi to ensure all the latest packages are installed ```bash sudo apt update sudo apt upgrade ``` ### Set up your machine peripherals 1. **Connect the webcam**: Connect the webcam's USB lead to any USB port on your Pi. The webcam transmits video data to the Raspberry Pi through the USB interface. 2. **Connect the piezo buzzer**: The passive piezo buzzer can be controlled via a GPIO pin on the Raspberry Pi. Refer to the following wiring diagram to connect the Raspberry Pi to the piezo buzzer. - Pin 35 (GPIO 19) to Positive - Pin 39 (GND) to Negative Now that we have physically connected our hardware components, let's configure the software in the next section. ## 6. Delivery Plan - Software Setup ### Configure your machine 1. In [the Viam app](https://app.viam.com/fleet/locations) under the **LOCATIONS** tab, create a machine by typing in a name and clicking **Add machine**. 2. Click **View setup instructions**. 3. Install `viam-server` on the Raspberry Pi device that you want to use to communicate with and control your camera and piezo buzzer. Select the `Linux / Aarch64` platform for the Raspberry Pi, and leave your installation method as [`viam-agent`](https://docs.viam.com/how-tos/provision-setup/#install-viam-agent). 4. Use the `viam-agent` to download and install `viam-server` on your Raspberry Pi. Follow the instructions to run the command provided in the setup instructions from the SSH prompt of your Raspberry Pi. 5. The setup page will indicate when the machine is successfully connected. ### Configure your Raspberry Pi board To access the GPIO pins, let's add our Raspberry Pi board to our machine in the Viam app. 1. In [the Viam app](https://app.viam.com/fleet/locations), find the **CONFIGURE** tab. 2. Click the **+** icon in the left-hand menu and select **Component**. 3. Select `board`, and find the `raspberry-pi:rpi4` module. This adds the module for working with the Raspberry Pi 4's GPIO pins. Leave the default name `board-1` for now. 4. Notice adding this module adds the board hardware component called `board-1`. The collapsible card on the right corresponds to the part listed in the left sidebar. 5. Click **Save** in the top right to save and apply your configuration changes. ### Configure your USB webcam Make sure your webcam is already connected to the Pi. 1. In [the Viam app](https://app.viam.com/fleet/locations) under the **CONFIGURE** tab, click the **+** icon in the left-hand menu and select **Component**. 2. Select `camera`, and find the `webcam` module. This adds the module for working with a USB webcam. Leave the default name `camera-1` for now. 3. Notice adding this component adds the webcam hardware component called `camera-1`. From the **Attributes** section of the panel, select a `video_path`. 4. Click **Save** in the top right. This may take a moment to apply your configuration changes. 5. At the bottom of the `camera-1` panel, expand the **TEST** section to ensure you have configured the camera properly and see a video feed. ### Configure your ML model service 1. In [the Viam app](https://app.viam.com/fleet/locations) under the **CONFIGURE** tab, click the **+** icon in the left-hand menu and select **Service**. 2. Select `ML model`, and find the `TFLite CPU` module. This adds support for running TensorFlow Lite models on resource-constrained devices. Name the service `people`. 3. In the new ML Model service panel, configure the service by setting the **Deployment** field to `Deploy model on machine`. 4. In the **Model** field, select the `EfficientDet-COCO by viam-labs` model under the **Registry** tab. This model is specifically optimized for resource-constrained devices like the Raspberry Pi. 5. Your ML model configuration should now look like this: 6. Click **Save** to apply your configuration changes. ### Configure your vision service 1. In [the Viam app](https://app.viam.com/fleet/locations), navigate to the **CONFIGURE** tab. Click the **+** icon in the left-hand menu and select **Service**. 2. Select `vision`, then find and choose the `ML model` module. This adds a new vision service to your machine. Name the service `peopleDetector`. 3. In the new vision service panel, configure the service by selecting `people` in the **ML model** field. This links your vision service to the ML model service you set up in the previous step. 4. Click **Save** to apply your configuration changes. Note that this may take a moment to complete. 5. At the bottom of the `peopleDetector` panel, expand the **TEST** section to verify your configuration. You should see bounding boxes and labels around detected objects. 6. To control the sensitivity of object detection, adjust the `Minimum confidence threshold` under the **Attributes** field. ### Configure your piezo buzzer You can manually and programmatically use the GPIO pins of the [`board`](https://docs.viam.com/components/board/) component to send PWM signals to control your buzzer. However, to streamline this next step, let's use a prebuilt module. 1. In [the Viam app](https://app.viam.com/fleet/locations) under the **CONFIGURE** tab, click the **+** icon in the left-hand menu and select **Component**. 2. Select `generic`, and find the `buzzer:piezo` module. This adds the module for controlling your buzzer. Name the component `piezo`. 3. In the new `piezo` panel, configure your component by adding the following attributes in the **CONFIGURE** field. This tells your piezo component to use a specific pin (physical pin 35) on a specific board (called `board-1` in the Viam app) that you wired in a previous step. ```json { "piezo_pin": "35", "board": "board-1" } ``` 4. Select your board `board-1` from the **Depends on** field. 5. Click **Save** to apply your configuration changes. This may take a moment. 6. Since we are using a generic component, let's test it out under the **CONTROL** tab. Find your `piezo` component on this page. Expand the **DO COMMAND** field, input the following code, and hit **Execute**. ```json { "sound_buzzer": { "frequency": 1200, "duration": 1.5, "duty_cycle": 0.7 } } ``` You are executing a [`DoCommand`](https://docs.viam.com/components/generic/#api) on a generic component that has been predefined to accept parameters such as `frequency`, `duration`, and `duty_cycle`. 7. Experiment with different values in those fields to see what happens when you execute the command again. If you have extra time, try sending the following Do command. ```json { "play_harry_potter": {} } ``` ## 7. Delivery Plan - Hands-on Experiment ### Program your people detector At this point, the Vision Service relies on the ML Model Service to analyze visual data. It uses pre-trained machine learning models, such as `EfficientDet-COCO`, to perform tasks like object detection. Currently, many objects are being detected in the camera feed. In this section, we'll add code to alert us when a `person` is detected. And if a person is detected, the piezo buzzer will sound. ### Add a control logic module Now let's add a control logic module to our Viam machine. 1. In [the Viam app](https://app.viam.com/fleet/locations), go to the **CONFIGURE** tab. Click the **+** icon in the left-hand menu and select **Component or service**. 2. Find, and add the module called `people-piezo`, and update the name of the module to `controller`. 3. In the new `controller` service, find the corresponding panel on the right, and add the required JSON configuration. The name of the camera, generic component, and vision service can be found in the Viam app interface. ```json { "camera": "camera-1", "generic": "piezo", "vision": "people-detector" } ``` 4. **Save** and apply your changes. 5. Test the new control module by navigating to the **CONTROL** tab. Find the `people-detector` service and observe what happens when a person is detected. ## 8. Additional resources and troubleshooting ### Additional resources - The website [pinout.xyz](https://pinout.xyz/) is a helpful resource with the exact layout and role of each pin for Raspberry Pi. When working with Viam, make sure to reference the physical pin numbers, and not the GPIO numbers listed on `pinout.xyz`. - [ML model service](https://docs.viam.com/services/ml/) - [Vision service](https://docs.viam.com/services/vision/) ### Common pitfalls and troubleshooting guidance 1. **Security** - Do not share your API credentials publicly. Sharing this information could compromise your system security by allowing unauthorized access to your machine, or to the computer running your machine. 2. **Flashing firmware** - Make sure you are using a 5V 5A (25W) power supply for the Raspberry Pi. - Participants must remember the `hostname` and `username` they set while flashing their Raspberry Pi, as they will need this when they SSH into the Pi. - To save time, instructors can flash all the Pis ahead of time with pre-determined credentials and share the credentials with participants during the workshop. Each Pi should have a unique `hostname` to avoid conflicts on the shared local network, such as `-detection` or `-detection` if they are working in groups. 3. **Working with actuators** - **Disabling a resource**: In a group setting, working with piezos can be fun, but also potentially disruptive. Learners can disable the resource by toggling off `controller`. - **Alternative actuators**: A buzzer is just one type of actuator, but the possibilities are endless. 4. **Configuring machine and peripherals** - If any problems occur while setting up the machine and peripherals in [the Viam app](https://app.viam.com/), check under the **LOGS** tab to see what might be going wrong. ## 9. Post-Workshop Resources ### Group discussion topics 🗣️ Review suggested topics for discussion at key points during the workshop: - What were the biggest challenges participants faced during the workshop (e.g., wiring, coding, debugging)? - Share troubleshooting experiences and how participants resolved issues. - How can object detection be used in homes, schools, or offices? - What additional features could improve the system (e.g., alerts, cloud integration, mobile app)? ### Quiz questions ❓ Review the suggested quiz questions below to evaluate participants' understanding of key concepts. These questions can be used to assess knowledge either during or after the workshop. 1. **Hardware Assembly** - What is the purpose of setting a custom `hostname` during the Raspberry Pi OS configuration? - What is the primary purpose for using PWM (Pulse Width Modulation) to control a passive piezo buzzer? 2. **Software Setup** - Why is it important to connect your machine to the Viam app during setup? ## Next-level projects Here are some ideas for expanding and enhancing the current project. - Explore other [object detection sensors to use on your next build](/content/post/object-detection-sensors-to-use-on-your-next-build-2024/index.html) - Explore other ML models like [a YOLO model](/content/post/guide-yolo-model-real-time-object-detection-with-examples/index.html) - [Deploy a model from Hugging Face](/content/post/smarter-vision-systems-hugging-face-viam/index.html) onto your Viam machine - Replace or add actuators as described in the previous section.