Set up people detection notifications in Home Assistant

Set up people detection notifications in Home Assistant

1. Overview

Home Assistant is one of the most popular smart home platforms in the world with a vast ecosystem of integrations to monitor and automate the devices around our homes. The Viam community integration makes it possible to collect images and sensor data from those existing connected devices, train custom machine learning models, and run them locally as part of an automation workflow.

In this codelab, you'll learn how to install the integration through the Home Assistant Community Store (HACS) and use a vision service to detect people from a camera connected to Home Assistant.

What You'll Build

Prerequisites

What You'll Need

What You'll Learn

2. Configure your machine

Create your machine

  1. In the Viam app under the LOCATIONS tab, create a machine by typing in a name and clicking Add machine.
  2. Click View setup instructions.
  3. To install viam-server on the Raspberry Pi device that you want to use to communicate with and control your webcam, select the Linux / Aarch64 platform for the Raspberry Pi, and leave your installation method as viam-agent.
  4. Use the viam-agent to download and install viam-server on your Raspberry Pi. Follow the instructions to run the command provided in the setup instructions from the SSH prompt of your Raspberry Pi.
  5. The setup page will indicate when the machine is successfully connected.

Add an ML model service

  1. In the Viam app, click the + icon in the left-hand menu and select Service, and then ML model.
  2. Search for a module called TFLite CPU. Then click Add module, and Create a new ML Model service called mlmodel-1. This provides the ability to run Tensorflow Lite, a.k.a LiteRT, models on the machine.
  3. Notice this creates two new items in the left sidebar. The first is your new ML Model service called mlmodel-1, and the second is the tflite_cpu module from the Registry.
  4. In the mlmodel-1 panel, click "Select model" in the Model section, search for "COCO" under the Registry tab, and select the "EfficientDet-COCO" model from the list. This is a general object detection model that is very efficient, as the name implies, for resource-constrained devices like a Raspberry Pi.
  5. Click Save in the top right to save and apply your configuration changes.

Add a vision service

  1. In the Viam app, click the + icon in the left-hand menu and select Service, and then vision.
  2. Search for a module called mlmodel. Then click Add module, and Create a new Vision service called vision-1.
  3. Notice adding this service adds the vision service called vision-1. From the ML Model section of the panel, select mlmodel-1.
  4. Save your changes in the top right and wait a few moments for the configuration changes to take effect.

3. Add Viam integration to Home Assistant

As a community integration, Viam is available through the Home Assistant Community Store (HACS). After getting started with HACS, you can add the Viam integration from the store.

  1. From the HACS dashboard in Home Assistant, search for Viam at the top, and select the top result.
  2. The integration documentation will be displayed. Click on the blue "Download" button in the bottom right to install the code onto your Home Assistant device.
  3. Once the installation is complete, go to the "Settings" page and select "Devices & services" to view the integrations dashboard.
  4. Click on "Add Integration", search for Viam, and select the top result.
  5. In the configuration form, enter the API key ID and API key for the machine you set up earlier. You can find those values under the "Connect" tab in the Viam app, in the API keys section.
  6. Click "Submit" to create the connection between Home Assistant and the Viam app. Click "Finish" when the success message appears to confirm the setup is complete.
  7. The Viam integration should now appear on your Integrations Dashboard.

4. Create People Detection Automation

The Viam integration provides a few services that can be used in Home Assistant automations. For this codelab, you'll use the object detection service to intelligently notify when people have been detected by existing cameras connected to your home.

  1. In the Home Assistant app, go to the "Settings" page and select "Automations & scenes" to view the Automations Dashboard.
  2. Click "Create Automation" and select "Create new automation" from the options in the modal.
  3. The "New automation" page will include sections for "When", "And if", "Then do" to configure for the workflow.
  4. Click "+ Add Trigger" to open the selection modal, then search for "Time pattern" and select the top result.
  5. Enter "/30" in the "Seconds" field to trigger the automation every 30 seconds. You can configure different patterns if you wish to check more or less frequently.
  6. Click "+ Add Action" to open the selection modal, then search for "Detect object" and select the top result.
  7. In the "Viam machine" field, select the configured machine from earlier. Enter the name of the vision service, "vision-1", in the "Detector Name" field. Check the "Camera Entity" box and select the connected camera you would like to use for this automation. Then set the "Response variable" to "people_detections".
  8. Click "+ Add Action" to open the selection modal, then search for "Condition" and select the top result.
  9. Select "Template" for the "Condition type" and enter the following into the "Value template" field:
    { { people_detections.detections|count > 0 } }
    ```

10. Click "+ Add Action" to open the selection modal, then search for "Persist" and select the top result. 
11. Click the action menu (three vertical dots) on the new "Persistent Notifications: ‘Create'" card, and select "Edit in YAML" to display the YAML editor for creating templated notifications. Replace the `data: {}` line with the following code:

```yaml
    data:
      title: People Alert
      message: >
        Detected {{ people_detections.detections|join(", ", "name")|default("nothing", true) }}: 
    ```

This will create a notification with a list of detected people and the associated image. 
12. Click the "Save" button in the bottom-right corner to enter a name for the automation and complete the setup.

## 5. Next Steps

Congratulations, you now have automated people detection for your smart home! Building on this experience you can do the following to make it even smarter:

- [automatically collect images](https://github.com/HipsterBrown/viam-home-assistant-integration?tab=readme-ov-file#viamcapture_image) or [sensor data](https://github.com/HipsterBrown/viam-home-assistant-integration?tab=readme-ov-file#viamcapture_data) from your Home Assistant devices in Viam
- use that data to [train and deploy a custom vision model](https://docs.viam.com/how-tos/train-deploy-ml/) to detect and classify other objects besides people, or even identify specific people
- [visualize the collected sensor data](https://docs.viam.com/how-tos/sensor-data-visualize/) using tools like Grafana

## What You Learned

- How to use a vision service in Viam
- How to install a community integration in Home Assistant
- How to configure an automation in Home Assistant

## Additional Viam resources

- [Viam documentation](https://docs.viam.com/)
- [Viam how-to guides](https://docs.viam.com/how-tos/)
- [Viam Discord community](http://discord.gg/viam)