Detect objects by color | Object detection

Detect objects by color

Use the color_detector vision service when you need to detect objects that stand out by color. It runs a heuristic hue-match on every frame with no ML model, no training data, and negligible compute cost.

Typical use cases: detecting colored markers, sorting parts by color on a conveyor, finding a red stop button, spotting a green plant against soil. If the target objects share a distinct hue and the background does not, color detection is the simplest possible vision pipeline.

When to use color detection vs ML detection

Pick color detection when:

Pick ML detection when:

You can also combine them: a color detector for a known marker alongside an ML detector for people and obstacles.

Limits of color detection

The color_detector model runs a heuristic on HSV color values. It cannot detect:

1. Pick a target color

Get the actual hex color of your target as the camera sees it, not the color you think it is. Lighting, white balance, and the camera sensor all shift color values.

The practical approach:

  1. Configure the camera on your machine and navigate to its Test panel in the Viam app.
  2. Take a screenshot of the camera feed with the target object clearly visible.
  3. Use a pixel-level color picker on the screenshot (browser extensions like Color Picker for Chrome work well).
  4. Sample three or four pixels on different parts of the object. Note the hex values.
  5. Pick one as your detect_color. If the values vary, you will need a wider hue_tolerance_pct in step 3.

2. Add the color_detector vision service

  1. Open the CONFIGURE tab in the Viam app.
  2. Click the + icon and select Blocks.
  3. In the search field, type color detector and select the vision/color_detector result.
  4. Click Add to machine, name the service (for example, red_detector), and click Add to machine again to confirm.

3. Configure the detector

{
  "name": "red_detector",
  "api": "rdk:service:vision",
  "model": "color_detector",
  "attributes": {
    "detect_color": "#C43131",
    "hue_tolerance_pct": 0.07,
    "segment_size_px": 200,
    "label": "red",
    "camera_name": "camera-1"
  }
}

See the color_detector reference for every attribute including the optional saturation_cutoff_pct and value_cutoff_pct.

Save the configuration.

4. Verify in the Control tab

  1. Navigate to the CONTROL tab.
  2. Click your vision service.
  3. In the Camera dropdown, select the camera whose feed you want the detector to run on. Bounding boxes appear as an overlay on the live camera feed and refresh automatically.

Point the camera at the target color. Bounding boxes should appear around regions of that hue. If you see:

Tuning is quick: each change takes a viam-server reload (a few seconds) to apply.

5. Use detections in code

Color detections come through the same vision service API as ML detections. Any code that works with GetDetections works with a color detector.

import asyncio

from viam.robot.client import RobotClient
from viam.services.vision import VisionClient

async def connect():
    opts = RobotClient.Options.with_api_key(
        api_key="YOUR-API-KEY",
        api_key_id="YOUR-API-KEY-ID",
    )
    return await RobotClient.at_address("YOUR-MACHINE-ADDRESS", opts)

async def main():
    machine = await connect()
    detector = VisionClient.from_robot(machine, "red_detector")

detections = await detector.get_detections_from_camera("camera-1")

for d in detections:
        print(f"{d.class_name}: confidence {d.confidence:.2f}, "
              f"box ({d.x_min}, {d.y_min}) to ({d.x_max}, {d.y_max})")

await machine.close()

if __name__ == "__main__":
    asyncio.run(main())

package main

import (
  "context"

"go.viam.com/rdk/logging"
  "go.viam.com/rdk/robot/client"
  "go.viam.com/rdk/services/vision"
)

func main() {
  ctx := context.Background()
  logger := logging.NewLogger("detector")

machine, err := client.New(ctx, "YOUR-MACHINE-ADDRESS", logger,
    client.WithDialOptions(client.WithEntityCredentials(
      "YOUR-API-KEY-ID",
      client.Credentials{
        Type:    client.CredentialsTypeAPIKey,
        Payload: "YOUR-API-KEY",
      })),
  )
  if err != nil {
    logger.Fatal(err)
  }
  defer machine.Close(ctx)

detector, err := vision.FromProvider(machine, "red_detector")
  if err != nil {
    logger.Fatal(err)
  }

detections, err := detector.DetectionsFromCamera(ctx, "camera-1", nil)
  if err != nil {
    logger.Fatal(err)
  }

for _, d := range detections {
    bb := d.BoundingBox()
    logger.Infof("%s: score %.2f, box (%d,%d)-(%d,%d)",
      d.Label(), d.Score(), bb.Min.X, bb.Min.Y, bb.Max.X, bb.Max.Y)
  }
}

Use multiple color detectors

You can configure multiple color_detector services against the same camera, each tuned for a different color. This is the simplest way to tag parts by color: one detector per color, each with its own label.

"services": [\
  {\
    "name": "red_parts",\
    "api": "rdk:service:vision",\
    "model": "color_detector",\
    "attributes": {\
      "detect_color": "#C43131",\
      "hue_tolerance_pct": 0.07,\
      "segment_size_px": 150,\
      "label": "red",\
      "camera_name": "camera-1"\
    }\
  },\
  {\
    "name": "blue_parts",\
    "api": "rdk:service:vision",\
    "model": "color_detector",\
    "attributes": {\
      "detect_color": "#1C4599",\
      "hue_tolerance_pct": 0.07,\
      "segment_size_px": 150,\
      "label": "blue",\
      "camera_name": "camera-1"\
    }\
  }\
]

Your code queries each detector in sequence. For higher throughput, call them concurrently.

Troubleshooting

Service fails to start
Color looks right to me but the detector sees nothing

What your eye sees and what the camera records are often different:

Sample the actual on-screen color from a live camera screenshot and use that hex value. Adjust hue_tolerance_pct up if you still do not get detections.

Works in testing, fails in production

Color detection is sensitive to lighting changes. A system that works under controlled lab lighting often fails under sunlight, under fluorescent lights (which flicker at 50 or 60 Hz and can beat against camera frame rates), or under mixed light.

Fixes, in order of preference:

  1. Add consistent artificial lighting so the target color does not shift.
  2. Switch to an ML-based detector trained on images from the actual production environment.

Next steps