> ## Documentation Index
> Fetch the complete documentation index at: https://docs.guild.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation checks

> Assert conditions about an agent run's output with evaluation checks, including the deterministic outputMatches lexical check.

Evaluation checks let you assert conditions about the output of an agent run so you can measure quality automatically. Each check inspects a sample from a run and returns a pass, fail, or abstention.

Guild supports four check types. Three are structural — `hasToolCall`, `toolCallContains`, and `outputMatches` — and compare values directly. The fourth, `llmJudge`, asks a model to judge the output. Use a structural check when the correct answer is knowable without reasoning, and reserve `llmJudge` for open-ended judgments.

This page documents `outputMatches`.

## outputMatches

Use `outputMatches` to assert whether a specific substring is present or absent in the final output of an agent run. It is a deterministic lexical check: it compares strings directly rather than asking a model to judge the output. Unlike `llmJudge`, `outputMatches` does not require an LLM judge, which reduces latency and cost.

### Parameters

| Parameter       | Type    | Required    | Default | Description                                                                                                                                     |
| --------------- | ------- | ----------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `type`          | string  | Yes         | —       | Must be `"outputMatches"`.                                                                                                                      |
| `mode`          | string  | Yes         | —       | Either `"contains"` or `"notContains"`. `"contains"` passes when the substring is present; `"notContains"` passes when the substring is absent. |
| `value`         | string  | Conditional | —       | A literal, non-empty string to match. Provide exactly one of `value` or `referencePath`.                                                        |
| `referencePath` | string  | Conditional | —       | A dot-separated path pointing to a value in the sample's reference data. Provide exactly one of `value` or `referencePath`.                     |
| `caseSensitive` | boolean | No          | `true`  | When `false`, performs Unicode-aware case-insensitive matching by comparing case-folded (`casefold()`) values.                                  |

<Note>
  Exactly one of `value` or `referencePath` must be provided. `value` must be a non-empty string. `referencePath` must resolve to a non-empty string.
</Note>

### Match against a literal value

Assert that the output contains a literal substring:

```json theme={null}
{
  "type": "outputMatches",
  "mode": "contains",
  "value": "Order confirmed"
}
```

Assert that the output does not contain a substring:

```json theme={null}
{
  "type": "outputMatches",
  "mode": "notContains",
  "value": "stack trace"
}
```

### Match against reference data

Use `referencePath` to compare the output against a value stored in the sample's reference data. This lets you assert against per-sample expected values instead of hardcoding them:

```json theme={null}
{
  "type": "outputMatches",
  "mode": "contains",
  "referencePath": "expected.orderId"
}
```

<Warning>
  If `referencePath` does not resolve to a non-empty string, the check does not fail. It abstains with the code `SETUP_INVALID`. See [Aggregation](#aggregation).
</Warning>

### Case-insensitive matching

Set `caseSensitive` to `false` to match regardless of letter case. Matching is Unicode-aware and compares the `casefold()` representations of both values:

```json theme={null}
{
  "type": "outputMatches",
  "mode": "contains",
  "value": "order confirmed",
  "caseSensitive": false
}
```

### Aggregation

Results of `outputMatches` checks are aggregated by spec index and check type.

If a check cannot resolve its `referencePath`, it does not count as a pass or a fail. It is reported as an abstention under `abstained` and `abstain_reasons` with the code `SETUP_INVALID`.