> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ancha.io/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> These docs describe the Ancha CLI, npm package @ancha-labs/cli version 0.1.4, and the hosted workspace at app.ancha.io. Coding agents should start at https://docs.ancha.io/coding-agents. Run the CLI as `ancha` after `npm install -g @ancha-labs/cli`, or as `npx -y @ancha-labs/cli@0.1.4` without installing. Pass --json for machine-readable output, or set ANCHA_JSON=1. Every command ends with a `next:` line naming the next command. Exit codes: 0 pass, 1 a check failed or a comparison regressed, 2 usage, config or authentication error, 3 infrastructure error, 4 unimplemented endpoint with --strict.

# Checks

> Checks decide whether an attempt passed. State checks read the final world, trace checks read the agent's API calls, process checks read what happened between the agent and the people in the world, and judge checks ask a model to grade text.

A check is a named condition that Ancha evaluates when an attempt ends. An attempt passes when every required check passes. Without checks, an attempt is ungraded: it ran, but nothing says whether it did the right thing.

## Four kinds

| Kind | Reads | Use it for |
| - | - | - |
| `state` | The final state of the tools | Outcomes: a reply exists in the right thread, a file has the right content, an issue carries the right label, an incident is resolved. |
| `trace` | The agent's API calls, in order | Behavior: the agent filed an issue, or never wrote to the CRM. |
| Process checks | What the world's engine recorded between the agent and the people in it | Conversations: the agent asked the customer the right question, told the engineer the details, never leaked a confidential fact, and did things in the right order. `ancha run` and `ancha check` only. |
| `judge` | The agent's API calls and final Slack messages, graded by a model | Qualities that exact conditions cannot express, such as whether a reply was correct and polite. `ancha run` and `ancha check` only. |

Prefer state checks for outcomes. They pass however the agent got there, while trace checks depend on which endpoint it chose.

## Examples

In a scene file, checks are YAML:

```yaml scene.yaml theme={null}
checks:
  - id: answered
    state:
      slack.message:
        channel: $slack_support
        from: $slack_bot
        in_thread_of: $slack_question
        text_contains: "Reports"
  - id: filed
    trace: { called: "POST /repos/acme/orders/issues" }
  - id: read_only_crm
    trace: { not_called: ["POST /crm/*", "PATCH /crm/*", "DELETE /crm/*"] }
  - id: asked_first
    before: [{ revealed: [laura.workspace] }, { called: "POST /repos/*/issues" }]
  - id: helpful
    judge: "The reply tells the customer where to find the CSV export and how long it takes."
pass: [answered, filed, read_only_crm, asked_first]
```

In the hosted console and in playground task files, the same checks are a JSON list on each task:

```json theme={null}
[
  {"id": "filed", "trace": {"called": "POST /repos/acme/orders/issues"}},
  {"id": "answered", "state": {"slack.message": {"channel": "$slack_support", "from": "$slack_bot", "in_thread_of": "$slack_question", "text_contains": "issues/"}}}
]
```

## Required checks

`pass` lists the IDs of the required checks. It defaults to every check. A check outside `pass` is still evaluated and reported, but it does not decide the outcome. Use that for advisory checks, such as a judge rubric you are still tuning.

## Trace patterns

A trace pattern is `"<METHOD> <PATH>"`, for example `"POST /api/chat.postMessage"`.

* The path is relative to the tool's base URL and has no query string: `/repos/acme/starter`, not the full URL.
* The method is compared without regard to case. `*` as the method accepts any method. The path must match exactly, including case.
* `*` in the path matches any run of characters, including `/`. `"POST /crm/*"` matches every POST under `/crm/`. The pattern `"*"` matches any call.
* Only the agent's own calls count. Actions taken by the world, such as a person posting in Slack, never match.

| Field | Passes when |
| - | - |
| `called` | Every listed pattern matches at least one call. |
| `not_called` | No call matches any listed pattern. |

With both fields set, both must hold.

> [!TIP]
>
> Use `*` as the method when the verb does not matter. Slack's official SDKs call every Web API method with `POST`, while plain `fetch` code often uses `GET`, and Slack accepts both. The starter scene's checks are written this way, for example `* /api/*.*`. Check the agent's calls with `ancha explain` or in `trace.jsonl` when a trace check surprises you.

## State checks

| Check | Passes when |
| - | - |
| `slack.message` | At least `count_gte` messages (default 1) in a channel match all the conditions given: author, a thread, a case-insensitive substring. |
| `github.file_at` | A file at a branch, commit or `production` contains, or equals, the given text. |
| `github.pull` | A pull request is merged, or not. |
| `github.production` | Production is exactly a given commit. |
| `github.issue` | Issues in a repository match a title, body, labels, state and count. |
| `datadog.monitor` | A monitor's state is `OK` or `Alert`. |
| `datadog.downtime` | An active downtime covers a monitor with enough time left and a bounded total length. |
| `pagerduty.incident` | An incident has a status, an exact set of assignees, or was acknowledged or resolved within a time. |
| `fault_resolved` | A living-world fault is resolved. |

State checks count every record in the final state, including starter data. Set `from` on a `slack.message` check so a message from the starter history cannot satisfy it. See [Checks reference](https://docs.ancha.io/reference/checks.md) for every field.

## Process checks

A scene can hold [people](https://docs.ancha.io/reference/test-file.md#people): a customer who holds back details until asked, an engineer who knows the workaround, a manager who decides the priority. A model writes what they say, but the world's engine decides who receives a message, which facts a person may share, what they decide and when they answer, and it records every one of those events. Process checks read those records.

| Check | Passes when |
| - | - |
| `revealed` | The agent asked, and the owner of each fact shared it. |
| `knows`, `told` | A person knows each fact, from the agent or from someone else. |
| `decided` | A person made a decision, with a given option. |
| `message` | At least one message matching a selector reached a person or a place. |
| `never` | No message matched the selector, for example a confidential fact never reached the customer. |
| `before` | Milestones happened in order: a call, a message, or the first time a predicate held. |
| `within` | A milestone happened within a time limit of the start or of another milestone. |

Because the checks read the engine's records and never the model's text, a person who writes something unexpected cannot pass or fail an attempt by wording alone. A milestone that never happened fails `before` and `within`, so an attempt that did nothing cannot pass them. See [Process checks](https://docs.ancha.io/reference/checks.md#process-checks).

## Judge checks

A judge check is a rubric in plain language. After the attempt, Ancha sends a model a bounded copy of the evidence: the agent's API calls to Slack, GitHub, Datadog and PagerDuty with their requests and responses, and the final Slack messages, channels and users. The model returns passed or failed with a reason and the trace entries it relied on. Credentials, the scene's seed and the other checks are never sent.

* Judge checks need `OPENROUTER_API_KEY` in the environment of the `ancha run` or `ancha check` command. Ancha calls the model through OpenRouter. The agent does not get the key unless you list it in `agent.pass_env`.
* Set `judge.model` in `ancha.yaml` to the OpenRouter model you want the judge to use.
* Without a key, judge checks are skipped. A skipped check never passes, so a required judge check fails the attempt. Keep advisory rubrics out of `pass`.
* If the model call fails, the trial is marked invalid instead of failed.
* Judge checks run in `ancha run` and `ancha check` only. The hosted console and playground tasks accept `state` and `trace` checks.

## Refs

Checks name records with refs such as `$slack_support`, not with generated IDs. Refs come from the world's starter data or from your scene's `seed`, and people from its `people`. Saving a task in the hosted console checks every ref against its world. See [Worlds](https://docs.ancha.io/concepts/worlds.md#naming-records-with-refs).

## Results

Every check result has the same fields:

| Field | Meaning |
| - | - |
| `id` | The check's ID. |
| `required` | Whether it is in `pass`. |
| `passed` | The result. |
| `expected` | What the check wanted, in words. |
| `actual` | What the world held or what the agent did. |
| `explanation` | One sentence combining the two. |
| `evidence` | Sequence numbers of the trace entries behind the result. |

`ancha explain` prints these for failed checks, `ancha check --json` prints them all for a live world, and each trial's `checks.json` holds them all.


## Related topics

- [Checks reference](https://docs.ancha.io/reference/checks.md)
- [Lob](https://docs.ancha.io/tools/lob.md)
- [GitHub](https://docs.ancha.io/tools/github.md)
- [Troubleshooting](https://docs.ancha.io/troubleshooting.md)
- [Salesforce](https://docs.ancha.io/tools/salesforce.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.