Top computer vision projects to explore in 2024 | Viam

Top computer vision projects to explore in 2024

Computer vision applications are all around us, from unlocking your iPhone with your face to autonomous vehicles maintaining a safe distance from cars and pedestrians. With the rapid advances of AI, it’s a quickly growing field of engineering with applications in wide-ranging industries. If you’re just dipping your feet in this space, it helps to get some computer vision projects under your belt to build your skills while developing a deeper understanding of the technology. These projects are even more fun when you can connect the machine learning (ML) models used for computer vision projects to real-world devices that can perform different tasks in your physical environment. Combining online open-source platforms and Viam’s custom vision service, you can add ML to many devices you likely already have around your home. We’ve gathered 10 computer vision project ideas from the Viam community, complete with step-by-step instructions, so you can get your hands dirty and start building. Let’s take a look.

Beginner computer vision projects

1. Hugging Face model deployment for enhanced vision systems

Skills targeted: Model deployment, Hugging Face integration, and practical AI skills

Hardware you’ll need:

Start experimenting with ML applications using Hugging Face’s pre-trained models integrated with Viam's vision services through our registry. By embedding these ML models into your system, you’ll be ready for advanced image analysis and pattern recognition with minimal setup.

Steps to build:

  1. Install ‘viam-server’ and add a camera component in the Viam app.
  2. Add a YOLO vision service module (choose from YOLOv5 or YOLOv8) from our registry and configure it in your machine’s setup. Use our vision API to start writing script. See the full setup tutorial from Hugging Face.
  3. Now comes the fun part: choosing a model based on your use case. Select and integrate any compatible Hugging Face YOLO model with the vision service—from categorizing your shoe collection to tracking machines on a construction site.
  4. With your system set up, you can now test and visualize detections. A good place to start is using Viam's transform camera to display bounding boxes and then verify real-time object detection in the Viam app.

2. Object tracking with a mobile rover (like SCUTTLE)

Follow a colored object with a rover - YouTube

Skills targeted: Color detection, robotics integration, and control systems

Hardware you’ll need:

Let’s build a robot with color detection capabilities! Your robot will be able to follow an object of a specific color—whether it’s a red soccer ball like in the video or your chihuahua sporting a bright yellow sweater. This computer vision project is an entertaining, hands-on way to learn object tracking with a robot.

Steps to build:

  1. Assemble the SCUTTLE robot kit (or the Viam rover or another mobile rover) and attach and configure the webcam. Make sure the camera is connected to the rover’s board and then turn on power to the rover.
  2. Set up the Viam platform on your machine and configure color detection. This example uses #a13b4c, a reddish color, but you can specify any color you like.
  3. The detector will now draw boxes around objects of that color. Now test the detector from the CONTROL tab.
  4. It’s time to program your rover for color detection. Set up your code environment using Virtualenv. Now, connect Python code and follow the steps in the tutorial to process the camera feed and detect the specified color.
  5. Integrate the color detection results with the robot’s movement controls. Now, run the code again and test whether the rover follows an object of that color. Have fun with it!

3. Light automation with object detection

Skills targeted: Object detection, Python coding, and basic automation

Hardware you’ll need:

This computer vision project involves using a Python object detection library like OpenCV or YOLO to capture live video data and detect objects in a specific area.

Steps to build:

  1. Install ‘viam-server’ on your computer and add a new machine in the Viam app. Configure your webcam as an object detection camera. Then test the camera feed in the Control tab.
  2. Set up computer vision services and configure the ML model on your machine.
  3. Configure a transform camera to test bounding boxes for detected objects.
  4. Set up the Kasa smart plug and install the smart plug Python API.
  5. Write Python code to control your object detection device using the files in the GitHub repo for this project.
  6. Now, see how well this computer vision project works by moving in and out of the webcam’s line of sight to see if it detects you accurately.

4. Home energy optimization with AI

Skills targeted: Real-time monitoring, data analysis, and sustainability-focused engineering

Hardware you’ll need:

If you’ve ever wondered how energy usage in your house fluctuates based on how many people are home, this computer vision project is for you. This project takes it to another level by correlating your energy usage data to the presence of people in your house.

Steps to build:

  1. Set up the ESP32 camera with ‘viam-micro-server.’

  2. Configure the cloud server with ‘viam-server.’ Link the ESP32 as a remote part.

  3. Set up AI services on ‘viam-server.’ Configure the ML model and computer vision services and then integrate OpenAI's ChatGPT vision module.

  4. Implement person detection and energy monitoring. Use Viam Python SDK for detection and then integrate SolarEdge Monitoring API to track energy usage.

  5. Now you’re ready to collect and analyze the data! The project creator used ChatGPT to analyze the raw data to extract insights into energy usage patterns.

    CopyFrom the energy usage data for September 25, 2024, several insights can be drawn:

Key Time Insights: 6:00 AM to 7:00 AM: Significant energy increase, possibly due to morning routines.

5:00 PM: Highest energy usage of the day (3.75 kW) when everyone is home, likely related to evening activities.

Overnight: Consistent low energy usage, with minimal fluctuations, suggesting stable baseline consumption.

This data could help in identifying periods where energy-saving measures can be applied.


### 5. Helmet detection for job site safety

**Skills targeted:** Safety-focused AI, real-time detection, and public safety applications

**Hardware you’ll need:**

- A computer (running macOS or Linux) or single-board computer (SBC)
- A webcam

This computer vision project takes us to the field of job site safety by automating the process of monitoring workers’ use of personal protective equipment (PPE)—in this case, wearing a hard hat or helmet.

**Steps to build:**

1. [Connect](https://docs.viam.com/components/camera/webcam/) your webcam to your computer and [install](https://docs.viam.com/installation/) the Viam server on your device.
2. Set up Viam's [vision service](https://docs.viam.com/ml/vision/) and add the [YOLOv8 module](https://github.com/viam-labs/YOLOv8), then configure the [hard hat detection model](https://huggingface.co/keremberke/yolov8s-hard-hat-detection) from Hugging Face.
3. [Configure](https://docs.viam.com/tutorials/projects/helmet/#:~:text=save%20your%20changes.-,Configure%20the%20objectfilter%20module,-The%20physical%20camera) the ‘objectfilter’ module to process the camera feed and detect people with and without hard hats. 
4. Now, set up Viam’s [data management](https://docs.viam.com/data/cloud-storage/) features for [data capture](https://docs.viam.com/tutorials/projects/helmet/#:~:text=bounding%20boxes%20appear.-,Configure%20data%20capture%20and%20sync,-Viam%E2%80%99s%20built%2Din) to upload images of people without hard hats to the cloud.
5. The final step is to create a notification system. Write a serverless [function](https://docs.viam.com/program/functions/) to [send email notifications](https://docs.viam.com/tutorials/projects/helmet/#:~:text=save%20your%20changes.-,Set%20up%20email%20notifications,-Triggers%20allow%20you) each time the system detects a worker who isn’t wearing a hard hat.

### 6. Custom facial verification system

**Skills targeted:** Facial recognition, security protocols, and model training

**Hardware you’ll need:**

- A computer
- Webcam

**Steps to build:**

1. Connect your webcam to your computer and [install](https://docs.viam.com/installation/) ‘viamserver.’ [Configure](https://docs.viam.com/components/camera/) the camera component in the Viam app.
2. Configure the ML [model service](https://docs.viam.com/services/ml/). Then set up two vision detectors—a [people detection](https://docs.viam.com/tutorials/projects/verification-system/#:~:text=train%20your%20own.-,Use%20an%20existing%20ML%20model,-The%20ML%20model) model and a [facial recognition](https://docs.viam.com/tutorials/projects/verification-system/#configure-a-facial-detector:~:text=facial%20recognition%20detector.-,Configure%20a%20facial%20detector,-We%20now%20have) model. 
3. Now you’ll add the module that can detect specific faces and arm or disarm the alarm based on that data. [Add](https://docs.viam.com/tutorials/projects/verification-system/#configure-a-facial-detector:~:text=the%20available%20attributes.-,Configure%20a%20verification%20system,-Now%20that%20you%E2%80%99ve) the verification system module to your machine and configure it to use the two vision detectors.
4. Finally, create the visual overlay by [configuring](https://docs.viam.com/components/camera/transform/) a transform camera to display the current state of the verification system on your live camera feed.

### 7. Pet photography and data filtering module

**Skills targeted:** Image filtering, quality control, and AI-based image capture

**Hardware you’ll need:**

- Computer
- Webcam or other camera
- Colored item for your pet to wear

**Steps to build:**

1. Connect your camera to your computer and [install](https://docs.viam.com/installation/) ‘viamserver’ on your device.
2. Add a [camera](https://docs.viam.com/components/camera/) component using the Viam app. Then, [add](https://docs.viam.com/tutorials/configure/pet-photographer/#:~:text=module%20from%20scratch.-,Download%20the%20colorfilter%20module,-Follow%20the%20instructions) the ‘colorfilter’ module from Viam and [configure](https://docs.viam.com/tutorials/configure/pet-photographer/#:~:text=the%20next%20section.-,Add%20as%20a%20local%20module,-Whether%20you%E2%80%99ve%20downloaded) it as a local module on your machine.
3. Configure both [data management](https://docs.viam.com/tutorials/configure/pet-photographer/#:~:text=a%20camera%20stream.-,Add%20the%20data%20management%20service,-To%20enable%20data) and [vision](https://docs.viam.com/tutorials/configure/pet-photographer/#:~:text=management%20service.-,Add%20the%20vision%20service,-To%20enable%20your) services from Viam to detect the colored item and then save the captured pet photos to the cloud.
4. Now, [configure](https://docs.viam.com/tutorials/configure/pet-photographer/#:~:text=color%20detector.-,Enable%20filtering%20by%20color,-With%20the%20vision) your camera to filter by color and then set up data capture. Now, you’re all set to [test](https://docs.viam.com/tutorials/configure/pet-photographer/#:~:text=of%20the%20screen.-,Test%20your%20color%20filter%20camera,-To%20test%20that) your pet photographer system and make any adjustments.

### 8. Interactive bedtime songs bot

**Skills targeted:** Human-centered design, sound integration, and real-time responsiveness

**Hardware you’ll need:**

- Computer with a webcam and speakers (or separate components that are already connected)

Have you ever wished you had a clone who could sing that same bedtime song over and over again to your kid? This project is for you—whether you’re a parent, aunt, uncle, grandma, or grandpa. By using computer vision to detect a specific toy, the bot will play a specific, prerecorded bedtime song.

**Steps to build:**

1. If you’re using external cameras or speakers, connect them to your computer. Then [install](https://docs.viam.com/installation/) ‘viamserver’ and the [Go Client SDK](https://pkg.go.dev/go.viam.com/rdk).
2. [Configure](/content/post/bedtime-songs-bot#:~:text=Configure%20your%20webcam%20to%20capture%20data/index.html) your webcam to capture data, and then set up Viam’s [data management](https://docs.viam.com/data/capture/#add-the-data-management-service) services to save that data for analysis.
3. Train the ML model to recognize specific toys.
4. [Record](/content/post/bedtime-songs-bot#:~:text=Record%20bedtime%20songs/index.html) the songs you want your bot to play for the kids.
5. [Program](/content/post/bedtime-songs-bot#:~:text=Program%20your%20bedtime%2Dsongs%20bot/index.html) your robot to play a specific song when it sees a particular toy, using Viam’s Go SDK.

### 9. Plant health monitoring with local LLM and ML

**Skills targeted:** Local LLM deployment, plant care analysis, and sustainable design

**Hardware you’ll need:**

- A computer (MacBook, Windows with WSL, or edge devices like Raspberry Pi)
- A camera (built-in webcam or external camera)

**Steps to build:**

1. Install ‘viamserver’ on your computer and create a new machine in the Viam app.
2. [Add](https://docs.viam.com/ml/vision/) a computer vision component and connect it to a Roboflow model for plant detection.
3. [Add](https://app.viam.com/module/viam-labs/speech) a speech interface component so you can interact with the AI assistant through speech.
4. [Connect](/content/post/practical-ai-local-llm-and-machine-learning-for-plant-care#:~:text=should%20I%20do%3F%E2%80%9D-,Enhance%20your%20LLM%20assistant%20with%20computer%20vision,-Now%20that%20we/index.html) computer vision to your LLM assistant.

### 10. DIY hydroponic gardening system with vision technology

**Skills targeted:** Smart agriculture and environmental sensors

**Hardware you’ll need:**

- PVC pipes for hydroponic setup
- Raspberry Pi 5
- BME280 and BME680 environmental sensors
- pH sensor
- Webcam

If you have a green thumb, or you’re an aspiring indoor gardener, this might be the solution you never knew you needed. **Steps to build:**

1. Build the hydroponic system by constructing a PVC pipe system and then setting up the germination station.
2. Set up ‘viam-server’ on Raspberry Pi and connect ESP32, sensors, and webcam.
3. [Configure](https://docs.viam.com/components/camera/) the camera, set up time lapse imaging, and [set up](https://docs.viam.com/components/sensor/) sensors for temperature, humidity, and pH monitoring.

### Conclusion

From automating lights, to monitoring plant health, or building a facial verification tool, each project we’ve covered gives you hands-on experience with real-world applications of this technology.