# Build a bedtime songs bot with a custom ML model

Tess Avitabile  
Engineering Director

## Introduction

When I started at Viam, Eliot Horowitz told me the best way to test the product is to try to automate something I do in my life with a robot. As a parent of a 3-year-old and a 1-year-old, I am often presented with a toy and asked to sing a song about it. When I was testing out Viam’s ML Model service, I came up with the idea of using machine learning to make my computer do this instead.

You can follow this tutorial to train your own machine learning model to make your own “bedtime songs bot” out of a personal computer.

## Get started

To make your own singing robot, you need the following hardware:

- A computer with a webcam, speakers, and the [Go Client SDK](https://pkg.go.dev/go.viam.com/rdk) installed. I used a Macbook, but you can use any PC with a Viam-compatible operating system that meets the above requirements.

## Train your ML model with pictures of toys

### Configure your webcam to capture data

In the [Viam app](https://app.viam.com/), add a new machine and follow the setup instructions to install viam-server on your computer and connect to the Viam app.

Once connected, navigate to your machine’s page in the app and click on the **CONFIGURE** tab.

First, add your personal computer’s webcam to your robot as a [camera](https://docs.viam.com/components/camera/) by creating a new component with type _camera_ and model _webcam_:

Click the **+** icon next to your machine part in the left-hand menu and select **Component**. Select the _camera_ type, then select the webcam model. Enter _cam_ as the name or use the suggested name for your camera and click **Create**.

In the configuration panel, click the video path field. If your robot is connected to the Viam app, you will see a dropdown populated with available camera names.

Select the camera you want to use. If you are unsure which camera to select, select one, save the configuration, and go to the **Control** tab to confirm you can see the expected video stream. On the **Control** tab, click on the dropdown menu labeled **camera** and toggle the feed on. If you want to test your webcam’s image capture, you can click on **Export screenshot** to capture an image.

Now, configure the [Data Management Service](https://docs.viam.com/data/capture/#add-the-data-management-service) to [capture data](https://docs.viam.com/data/capture/), so you can use the image data coming from your camera on your robot to train your ML model:

1. Click the **+** icon next to your machine part in the left-hand menu and select **Service**.
2. Select the _Data Management_ type, then either use the suggested name or specify a name for your data management service, for example _data-manager_.
3. Click **Create**.
4. Make sure both **Data Capture** and **Cloud Sync** are enabled.

Enabling data capture and cloud sync lets you capture images from your webcam, sync them to the cloud and, in the Viam app, easily tag them and train your own machine learning model.

Next, [configure Data Capture for your webcam](https://docs.viam.com/data/capture/#configure-data-capture-for-individual-components):

1. Find the camera component panel from the camera you previously configured.
2. Click **\+ Add method** in the **Data Capture Configuration** section.
3. Set the **Type** to _ReadImage_ and the **Frequency** to 0.333. This will capture an image from the camera roughly once every 3 seconds. Adjust this frequency as needed.
4. Select the **Mime Type** as _image/jpeg_:

### Capture data

Your webcam is now configured to automatically capture images when you are connected to your robot live on [the Viam app](https://app.viam.com/). Grab the toys or any objects you want the robot to differentiate between.

Try to capture at least 50 images that fit your criteria for each tag by holding the toys up to the camera while photos are being taken.

Go to the [**DATA tab**](https://app.viam.com/data/view?view=images) in the Viam app to see the images captured by your webcam.

When you’ve captured enough images, navigate back to the **Config** tab. Scroll to the card with the name of your webcam and click the power switch next to the **Data Capture Configuration** to **off** to [disable data capture](https://docs.viam.com/data/capture/#configure-data-capture-for-individual-components).

### Tag data

Head over to the [**DATA** page](https://app.viam.com/data/view) and select an image captured from your robot. In the **Tags** section, add tags for each of the puzzle pieces corresponding to their shapes. Ensure you label at least 50 images for each tag to facilitate model training.

#### Filter based on tags

Now that you’ve tagged the image data, you can [filter your images](https://docs.viam.com/data/view/) according to those tags.

### Train a model

After tagging and filtering your images, begin training your model by clicking the **Train Model** button, naming your model "shape-classifier-model", and selecting **Multi label** as the model type. Choose the tags you used to label your toys and click **Train Model**.

Read through our guide to [training a new model](https://docs.viam.com/ml/train-model/) for additional information.

## Use your ML Model to sing songs to your kids

### Configure your webcam to act as a shape classifier

[Deploy the model](https://docs.viam.com/ml/) to the robot and [configure a vision service classifier of model mlmodel](https://docs.viam.com/ml/vision/mlmodel/) to utilize the trained classifier.

### Record bedtime songs

Capture the audio files for the songs you want your bot to play, ensuring that the file names match the classifier tags. Save your audio files and use the Viam Go SDK to program your robot to play a specific song when it recognizes a corresponding toy.

### Program your bedtime-songs bot

Follow these instructions to set up the Go control code:

1. Navigate to your machine’s page in [the Viam app](https://app.viam.com/), and click on the **CONNECT** tab.
2. Select **Go** as the language and copy the generated code.
3. Add your API key and API key ID as environment variables in your development environment.

4. Use the following example code:

```bash
cd / 
touch play-songs.go 
vim play-songs.go
```

In `play-songs.go`, add code to initialize the speaker and play songs whenever the webcam detects a toy:  
Initial speaker code:

```go
func initSpeaker(logger logging.Logger) {  }
```

**Full example of `play-songs.go` available at [GitHub](https://github.com/viam-labs/bedtime-songs-bot/blob/main/play_songs.go)**

Now, as shown below, your smart bedtime songs bot knows to play a song whenever it sees a shape in front of the camera.

## Next steps

This project is just a start. You can expand upon the [configuration](https://docs.viam.com/build/configure/) of your bedtime-songs bot to further customize a robot that can entertain with [machine learning](https://docs.viam.com/ml/), the [vision service](https://docs.viam.com/services/), and more [components](https://docs.viam.com/components/) and [services](https://docs.viam.com/services/).

Have questions, or want to meet others working on robots? Join our [Community Discord](https://discord.gg/viam).