# Configure an mlmodel detector or classifier

The `mlmodel` vision service wraps a deployed ML model and exposes it through the standard [vision service API](https://docs.viam.com/reference/apis/services/vision/). At startup, the service reads the model’s tensor metadata and decides which of three roles the model can fulfill: classifier, detector, or 3D segmenter. It registers every role the model supports.

## Prerequisites

Before configuring an `mlmodel` vision service, you need:

#### 1. A trained or uploaded ML model

Add an existing model from the [registry](https://app.viam.com/registry) or [train one from your data](https://docs.viam.com/train/train-a-model/). The model must be TensorFlow Lite, TensorFlow, ONNX, or PyTorch.

#### 2. An ML model service running on your machine

Configure an [ML model service](https://docs.viam.com/vision/deploy-and-maintain/deploy-from-registry/) with an implementation that matches your model format (for example, `tflite_cpu`, `onnx-cpu`, `tensorflow-cpu`, or `torch-cpu`).

## Configure

1. Navigate to the **CONFIGURE** tab of your machine’s page.
2. Click the **+** icon next to your machine part in the left-hand menu and select **Blocks**.
3. In the search field, type `vision` or `mlmodel` and select the `vision/mlmodel` result.
4. Click **Add to machine**, enter a name for your service, and click **Add to machine** again to confirm.
5. In the **ML MODEL** section, select the ML model service your model is deployed on.
6. In the **DEFAULT CAMERA** section, select the camera the service should use by default for calls such as `GetDetectionsFromCamera`.
7. Adjust other attributes in the attributes table as applicable.

Add the vision service object to the services array in your JSON configuration:

```json

```

## Attributes

| Attribute                        | Type                  | Required?      | Description                                                                                                                                               |
|----------------------------------|-----------------------|----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| `mlmodel_name`                  | string                | **Required**   | The name of the [ML model service](https://docs.viam.com/vision/deploy-and-maintain/deploy-from-registry/) the vision service wraps.                   |
| `camera_name`                   | string                | Optional        | The default camera to use for calls such as `GetDetectionsFromCamera`, `GetClassificationsFromCamera`, and `GetObjectPointClouds`.                     |
| `default_minimum_confidence`    | number                | Optional        | Minimum confidence score (between `0.0` and `1.0`) applied to all output labels. Detections and classifications below this are filtered out. If unset, no filtering is applied.<br>Example: `0.6` |
| `label_confidences`             | object                | Optional        | Per-label confidence thresholds. Keys are label names and values are minimum confidence. When set, `label_confidences` overrides `default_minimum_confidence` for listed labels and other labels are filtered out.<br>Example: `{"DOG": 0.8, "CARROT": 0.3}` |
| `label_path`                    | string                | Optional        | Path to a labels file. Overrides the label file specified in the ML model service. The file is one label per line; line number (zero-indexed) is the class ID. |
| `remap_input_names`             | object                | Optional        | Map model input tensor names to the names the vision service expects. The service expects `image` for the input tensor. See [Tensor name requirements](https://docs.viam.com/reference/services/vision/mlmodel/#tensor-name-requirements). |
| `remap_output_names`            | object                | Optional        | Map model output tensor names to the names the vision service expects (`location`, `category`, `score` for detectors; `probability` for classifiers). See [Tensor name requirements](https://docs.viam.com/reference/services/vision/mlmodel/#tensor-name-requirements). |
| `xmin_ymin_xmax_ymax_order`     | array of int          | Optional        | Four-entry permutation indicating the order in which the model outputs bounding box coordinates. Use `[0, 1, 2, 3]` when the model outputs `[xmin, ymin, xmax, ymax]`. Use `[1, 0, 3, 2]` when the model outputs `[ymin, xmin, ymax, xmax]`. Common source of shifted or mirrored detections when using custom YOLO variants. |
| `input_image_mean_value`         | array of float        | Optional        | Per-channel mean values subtracted from each pixel before inference. Requires at least 3 values, one per color channel. Set this only when the model was trained with non-default input normalization. If unset, no mean subtraction is applied.<br>Example: `[127.5, 127.5, 127.5]` |
| `input_image_std_dev`           | array of float        | Optional        | Per-channel standard deviation values. Each pixel is divided by this after mean subtraction. Requires at least 3 values, all non-zero. Set this only when the model was trained with non-default input normalization. If unset, no division is applied.<br>Example: `[127.5, 127.5, 127.5]` |
| `input_image_bgr`               | bool                  | Optional        | Set to `true` if the model expects BGR channel order instead of RGB. If detections have wrong colors or all labels appear at once, try flipping this.<br>Default: `false` |

## Tensor name requirements

The vision service expects specific tensor names from the wrapped ML model:

| Service role   | Input tensor | Output tensors              |
|----------------|--------------|-----------------------------|
| Detector       | `image`      | `location`, `category`, `score` |
| Classifier     | `image`      | `probability`                |

If your model uses different tensor names, set `remap_input_names` and `remap_output_names` to bridge them:

## Test your detector or classifier

Test an `mlmodel` vision service from the [Control tab](https://docs.viam.com/monitor/default-interface/#web-ui), with images in the cloud, or with code.

### Live camera footage

1. Open your machine in the Viam app and click the vision service’s **Test** area, or navigate to the **CONTROL** tab and select the vision service.
2. In the **Camera** dropdown, select the camera whose feed you want the vision service to run on. Detections above `default_minimum_confidence` appear as bounding boxes on the live camera feed and refresh automatically.

### Images in the cloud

If you have images stored in the [Viam Cloud](https://docs.viam.com/data/capture-sync/capture-and-sync-data/), you can run your classifier against them:
1. Navigate to the [**DATA** tab](https://app.viam.com/data/all) and click an image to open the expanded view.
2. Click the **Auto-prediction mode** icon in the image toolbar (or press `M`).
3. In the **Run model** panel, click **Choose ML model**, pick your model and version, then click **Run**.

### Code

The following examples get detections or classifications from a camera. Replace "camera-1" with the name of the camera you configured.

#### Tip

To fetch an image, detections, classifications, and point cloud objects in one round trip, use [`CaptureAllFromCamera`](https://docs.viam.com/reference/apis/services/vision/#captureallfromcamera). This is more efficient than separate calls and guarantees all results correspond to the same frame.

## Troubleshooting

##### Detections appear shifted or mirrored

The model’s output bounding box coordinate order does not match the vision service’s expected order. Set `xmin_ymin_xmax_ymax_order` to a permutation that matches your model. For example, a YOLO variant that outputs `[ymin, xmin, ymax, xmax]` needs `[1, 0, 3, 2]`.