> ## Documentation Index > Fetch the complete documentation index at: https://docs.ancha.io/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > These docs describe the Ancha CLI, npm package @ancha-labs/cli version 0.1.4, and the hosted workspace at app.ancha.io. Coding agents should start at https://docs.ancha.io/coding-agents. Run the CLI as `ancha` after `npm install -g @ancha-labs/cli`, or as `npx -y @ancha-labs/cli@0.1.4` without installing. Pass --json for machine-readable output, or set ANCHA_JSON=1. Every command ends with a `next:` line naming the next command. Exit codes: 0 pass, 1 a check failed or a comparison regressed, 2 usage, config or authentication error, 3 infrastructure error, 4 unimplemented endpoint with --strict. # Checks reference > Every field of state, trace, judge and process checks, with examples in YAML and JSON, and where each kind of check can be used. A check is an object with an `id` and exactly one check kind: `state`, `trace`, `judge`, or one of the process checks `revealed`, `knows`, `told`, `decided`, `message`, `never`, `before` and `within`. Scene files write checks in YAML under `checks:`. The hosted console and playground task files write the same objects as a JSON list. For how checks are evaluated, see [Checks](https://docs.ancha.io/concepts/checks.md). ```yaml scene.yaml theme={null} checks: - id: answered state: slack.message: { channel: $slack_support, from: $slack_bot, in_thread_of: $slack_question } - id: filed trace: { called: "POST /repos/acme/orders/issues" } pass: [answered, filed] ``` ```json Task checks (console and playgrounds) theme={null} [ {"id": "answered", "state": {"slack.message": {"channel": "$slack_support", "from": "$slack_bot", "in_thread_of": "$slack_question"}}}, {"id": "filed", "trace": {"called": "POST /repos/acme/orders/issues"}} ] ``` ## Where each kind works | Kind | Scene files (`ancha run`, `ancha check`) | Hosted console tasks | Playground task files | | - | - | - | - | | `state` | Yes | Yes | Yes | | `trace` | Yes | Yes | Yes | | `judge` | Yes | No | No | | Process checks | Yes | No | No | A playground task holds at most 20 checks. ## Common fields | Field | Type | Meaning | | - | - | - | | `id` | string | Unique name of the check. Results and `pass` refer to it. | | `state` | object | A condition on the final world state. | | `trace` | object | A condition on the agent's API calls. | | `judge` | string | A rubric for a model to grade. | | `revealed`, `knows`, `told`, `decided`, `message`, `never`, `before`, `within` | varies | A condition on what happened between the agent and the people in the world. See [Process checks](#process-checks). | `pass` is a scene-level list of required check IDs. It defaults to every check. ## Trace checks | Field | Type | Passes when | | - | - | - | | `called` | string or list | Every pattern matches at least one call by the agent that worked: a status below 400, not a coverage gap, no Slack `ok: false` and no GraphQL answer with only `errors`. | | `not_called` | string or list | No call by the agent matches any pattern, failed attempts included. | | `status` | `any` | With `called`: count every attempt, failed calls included. | Set at least one. With both, both must hold. A pattern is `" "`, or `"*"` for any call. The method ignores case, and `*` as the method accepts any method. The path is exact, relative to the tool's base URL, without the query string. `*` in the path matches any run of characters, including `/`. Only the agent's own calls count. ```yaml theme={null} - id: posted trace: { called: "* /api/chat.postMessage" } - id: read_before_write trace: { called: ["GET /repos/acme/orders/issues/13", "POST /repos/acme/orders/issues/13/comments"] } - id: no_crm_writes trace: { not_called: ["POST /crm/*", "PATCH /crm/*", "DELETE /crm/*"] } ``` `called` with a list does not check order. For order, use a `before` check with `called` milestones. ## State checks ### Slack message `slack.message` counts the messages in a channel that match every condition given and passes when there are at least `count_gte`. | Field | Required | Meaning | | - | - | - | | `channel` | Yes | Channel ref. | | `from` | No | Author ref. Set it so starter history cannot match. | | `text_contains` | No | Substring the message must contain, ignoring case. | | `in_thread_of` | No | Message ref. Only replies in that thread count, not the parent. | | `count_gte` | No | Minimum number of matching messages. Default `1`. | ```yaml theme={null} state: slack.message: { channel: $slack_support, from: $slack_bot, in_thread_of: $slack_question, text_contains: "Exports" } ``` ### GitHub | Check | Fields | Passes when | | - | - | - | | `github.file_at` | `[repo, revision, path]` plus exactly one of `contains` or `equals` | The file exists at that revision and contains, or equals, the text. Comparison is exact and case-sensitive. | | `github.pull` | `pull` (pull request ref), `merged` (boolean) | The pull request's merged state equals `merged`. | | `github.production` | `repo`, `commit` (refs) | Production is exactly that commit. | | `github.issue` | `repo`, then any of `title_contains`, `body_contains`, `labels`, `state`, and a count | Issues in the repository (not pull requests) match. With no count, at least one must match. | \| `github.permission` | `{ repo, user }`, then `is` | The user's effective access to the repository is `is`: `read`, `write`, `admin` or `none`. | `revision` is a branch name, a commit ref, `default` (the default branch) or `production`. `github.issue` fields: | Field | Meaning | | - | - | | `repo` | Repository ref. | | `title_contains`, `body_contains` | A string, or a list of strings that must all appear. Case-insensitive. | | `labels` | The issue has every one of these labels. | | `state` | `open` or `closed`. | | `count` | Exactly this many issues match. | | `count_gte`, `count_lte` | At least, or at most, this many. Not together with `count`. | ```yaml theme={null} - id: runbook_updated state: { github.file_at: [$github_orders, main, docs/runbook.md], contains: "verified health check" } - id: pr_merged state: { github.pull: { pull: $github_runbook_pr, merged: true } } - id: issue_filed state: { github.issue: { repo: $github_orders, body_contains: [CSV, export], labels: [bug], state: open, count: 1 } } ``` ### Datadog | Check | Fields | Passes when | | - | - | - | | `datadog.monitor` | monitor ref, `is`: `OK` or `Alert` | The monitor's current state is `is`. | | `datadog.downtime` | `monitor`, `remaining_gte`, `duration_lte`, optional `reject_active_unsafe: true` | An active, uncanceled downtime targets exactly that monitor, has at least `remaining_gte` left and lasts at most `duration_lte` in total. With `reject_active_unsafe`, it also fails if any active all-monitor or overlong downtime exists. | ```yaml theme={null} - id: monitor_ok state: { datadog.monitor: $datadog_orders, is: OK } - id: muted_briefly state: { datadog.downtime: { monitor: $datadog_orders, remaining_gte: 1m, duration_lte: 10m } } ``` ### PagerDuty `pagerduty.incident` selects one incident and checks at least one property. | Field | Meaning | | - | - | | `pagerduty.incident` | `{ ref: }`, or `{ service: , first: true }` for the earliest incident of a service, including ones created during the attempt. | | `status` | `triggered`, `acknowledged` or `resolved`. | | `assigned_to` | List of user refs: the exact final set of assignees, in any order. | | `acknowledged_within` | Duration from creation to the first real acknowledgment. | | `resolved_within` | Duration from creation to the first real resolution. | ```yaml theme={null} - id: resolved_fast state: pagerduty.incident: { service: $pagerduty_orders, first: true } status: resolved resolved_within: 5m ``` ### Fault resolved `fault_resolved: ` passes when a living-world fault declared in `world_rules.faults` is resolved. See [Test file](https://docs.ancha.io/reference/test-file.md#world-rules). ### Okta `okta.user: ` with `status`, `status_in` or `exists: false` passes when the user has that status, has one of the statuses in `status_in`, or no longer exists. ### Other tools The other tools have no state checks. Check what the agent did with trace checks. ## Process checks Process checks read what happened between the agent and the [people](https://docs.ancha.io/reference/test-file.md#people) in the world: who learned which fact, who decided what, which messages reached whom, and in what order. The world's engine records these as the attempt runs. A check reads the records, never the text a model wrote, and nothing a person writes can end an attempt. ```yaml theme={null} checks: - { id: asked_laura, revealed: [laura.workspace, laura.beta_on] } - { id: ben_got_the_details, knows: [ben, laura.beta_on] } - { id: laura_got_the_workaround, told: [laura, ben.workaround] } - { id: priority_decided, decided: [mike, severity, p1] } - { id: answered_in_her_thread, message: { to: laura, in_thread_of: $slack_laura_question, from: $slack_bot } } - { id: no_root_cause_to_customer, never: { to: laura, fact: ben.root_cause } } - { id: priority_before_label, before: [{ decided: [mike, severity, p1] }, { called: "POST /repos/*/issues/*/labels" }] } - { id: answered_fast, within: { event: { message: { to: laura, from: $slack_bot } }, of: start, limit: 30m } } ``` | Kind | Value | Passes when | | - | - | - | | `revealed` | `[., ...]` | The owner of each fact shared it with the agent because the agent asked for it. | | `knows` | `[, , ...]` | The person knows every listed fact, learned from the agent or from another person. | | `told` | `[, , ...]` | The same as `knows`, for checks that read better that way. | | `decided` | `[, ]` or `[, ,