ancha run

ancha run runs your agent against each scene in ancha.yaml, trials times, on a fresh world each time, evaluates every scene’s checks and exits with a code that says whether the agent passed.

ancha run [options] [scenes...]

Argument	Description
`scenes`	Scene YAML files (default: the globs in scenes: of ancha.yaml)

Option	Description
`--agent <cmd>`	Shell command that runs the agent (default: agent.command in ancha.yaml)
`--agent-url <url>`	Endpoint URL (overrides agent.url; requires endpoint setup/start configuration)
`-n, --trials <n>`	Trials per scene (default: trials in ancha.yaml, else 5)
`--parallel <n>`	Trials running at once (default: parallel in ancha.yaml, else command 2 / endpoint 1)
`--min-pass-rate <r>`	Fraction of trials that must pass, 0 to 1 (default: min_pass_rate in ancha.yaml, else 1)
`--seed <seed>`	Seed for ids and randomness
`--out <dir>`	Where run directories go (default: .ancha/runs)
`--strict`	Exit 4 if the agent hit an endpoint Ancha does not implement
`--detach`	Continue in a private worker after this command exits; observe with status/wait
`--compare <reference>`	Compare against a completed run id or saved baseline; any per-scene decrease fails
`--pass-env <name>`	Give the agent this variable from this shell (repeatable); it otherwise gets only a base set (PATH, HOME, locale…) plus Ancha’s and the tools’ variables
`--inherit-env`	Give the agent this shell’s whole environment, including any real credentials (the behavior before 0.1.2)
`--idempotency-key <key>`	Retry safely: the same key and the same run definition return the original run instead of starting another
`--on <target>`	Where the agent runs: local (a process on this computer), docker (a sealed container; needs agent.execution with an image in ancha.yaml) or cloud (your workspace: uploads this project, connects it there and runs each test there). Default: what ancha.yaml says
`--upload`	Send the finished run to your workspace, where the console shows it (also ANCHA_UPLOAD=1); a failed upload is a warning
`--upload-required`	Like —upload, but a failed upload fails the command

Example

ancha run

Output

running locally (worlds in a server this command starts)
run_started runId=run_20261031-234205_e9f7049f942a39d5 scenes=["starter"] trials=1 parallel=1 workspace=local:/path/to/project target=local
trial_started scene=starter trial=1 envId=0ykgpn5i
trial_finished scene=starter trial=1 passed=true valid=true warnings=[] exit=exited exitCode=0 realMs=79 failedChecks=[]

✓ starter  1/1 (100%)

agent not sealed: network open; environment limited to 9 variables from this shell (pass more with --pass-env NAME)
pass (minimum pass rate 100%)  run run_20261031-234205_e9f7049f942a39d5
next: ancha baseline save before

The first line says where the worlds run. After ancha login it reads

running locally (worlds in a server this command starts; logged in to https://app.ancha.io/t/<workspace>: --upload sends the results there)

: the login keeps the worlds on this computer and names the workspace that --upload sends results to. The last line names the next command. Pass scene files to run only those: ancha run scenes/support/scene.yaml. --agent "<command>" overrides agent.command for one run. Without ancha.yaml, run exits 2 and tells you to run ancha init.

How a run works

ancha run chooses a server for the worlds: the one --server-url, server.url or ANCHA_SERVER_URL names, else a private local server it starts for this run. A saved login does not move the worlds; it names the workspace for --upload. The first line names the server, for example running on hosted https://app.ancha.io/t/<workspace> (from ANCHA_SERVER_URL; --local to run here). --local keeps the worlds on this computer whatever is configured. See CLI overview.
For each scene and trial, it creates a fresh world from the scene’s seed and starts your agent with the task and the tool variables. See Environment variables.
When the agent exits or the scene’s timeout passes, it captures the final state and every API call, then evaluates the checks.
A trial passes when all required checks pass. A scene passes when its pass rate reaches min_pass_rate. The run passes when every scene passes and nothing regressed.

Results go to .ancha/runs/<run-id>/, and .ancha/runs/latest points at the newest completed run. See Attempts and runs.

What the agent gets

The agent runs as a process on this computer, in the project directory. It is not sealed: it can reach this computer’s network. Its environment is limited to a base set from your shell (PATH, HOME, USER, LOGNAME, SHELL, LANG, LC_*, TERM, TMPDIR, TZ, NODE_EXTRA_CA_CERTS, SSL_CERT_FILE, VIRTUAL_ENV, CONDA_PREFIX and PYTHONPATH), plus Ancha’s variables and the tools’ variables. Real credentials in your shell, such as GITHUB_TOKEN or DD_API_KEY, never reach it unless you name them:

ancha.yaml

agent:
  command: node agent.mjs
  pass_env: [OPENAI_API_KEY]   # names only; values come from your shell at run time

--pass-env NAME adds a name for one run and can be repeated. --inherit-env passes the whole shell environment, as 0.1.1 did, and prints a warning. The run summary says what the agent got, and ancha explain says how many variables were withheld. With agent.execution.mode: container, the agent gets only the names in agent.execution.env. See ancha.yaml.

Exit codes

Code	When
`0`	Every scene met `min_pass_rate` and nothing regressed.
`1`	A scene’s pass rate fell below `min_pass_rate`, or `--compare` found a scene or a check that got worse.
`2`	Usage, config or scene error, no agent command, or a hosted server that rejected the API key (`AUTH_INVALID`).
`3`	Infrastructure failed, or a trial was invalid because Ancha itself failed.
`4`	With `--strict`, the agent called an endpoint a replica does not implement.

When several apply, 3 wins over 4, and 4 wins over 1.

JSON output

ancha run --json prints NDJSON: run_started, then trial_started and trial_finished for each trial, then run_finished with the summary and the next commands. run_started and the summary carry workspace (the world server, or local:<project path>) and target (local for a process on this computer, docker for a container). See For coding agents for the fields, and ancha schema run_finished for the schema.

Compare with a baseline

ancha baseline save before
# change the agent
ancha run --compare before

Output

running locally (worlds in a server this command starts)
run_started runId=run_20261031-234206_09b422aebb589f2b scenes=["starter"] trials=1 parallel=1 workspace=local:/path/to/project target=local
trial_started scene=starter trial=1 envId=qbw16ffq
trial_finished scene=starter trial=1 passed=true valid=true warnings=[] exit=exited exitCode=0 realMs=77 failedChecks=[]

✓ starter  1/1 (100%)

compared with run_20261031-234205_7570556079db03cd: no regression
  starter: 0% → 100% (+100 pp); 1 → 1 trials
    improved: pagerduty_access  0/1 → 1/1
agent not sealed: network open; environment limited to 9 variables from this shell (pass more with --pass-env NAME)
pass (minimum pass rate 100%)  run run_20261031-234206_09b422aebb589f2b

--compare takes a baseline name or a run ID. Every scene and every required check is compared. A scene whose pass rate drops fails the run with exit 1, even when the overall pass rate is fine. So does a check whose pass rate drops, even when the scene’s rate holds, for example when one check went from 3/3 to 0/3 while another went from 0/3 to 3/3. Dropped checks are listed first:

Output

running locally (worlds in a server this command starts)
run_started runId=run_20261031-234925_9ba852ebe26b8eca scenes=["starter"] trials=1 parallel=1 workspace=local:/path/to/project target=local
trial_started scene=starter trial=1 envId=k7grx7pe
trial_finished scene=starter trial=1 passed=false valid=true warnings=[] exit=exited exitCode=0 realMs=76 failedChecks=["github_access"]

✗ starter  0/1 (0%)
    ✗ github_access  failed in 1/1 trials
      trial 1: The agent never made a call matching "* /*repos*". Its calls were: GET /api/conversations.list, GET /api/v1/dashboard, GET /services.

compared with run_20261031-234211_30f86efd56c0f25a: regressed
  ✗ dropped: starter github_access  1/1 → 0/1
  starter: 100% → 0% (-100 pp); 1 → 1 trials
agent not sealed: network open; environment limited to 9 variables from this shell (pass more with --pass-env NAME)
fail (minimum pass rate 100%)  run run_20261031-234925_9ba852ebe26b8eca
next: ancha explain run_20261031-234925_9ba852ebe26b8eca
Patterns are "<METHOD> <PATH>" matched against the service-relative path, and * is a wildcard.

Comparison needs the same scenes with the same content and checks; trial counts may differ. The summary’s comparison field holds baselineRunId, regressed, droppedChecks and a per-scene before, after and delta.

Coverage gaps and `--strict`

When the agent calls an endpoint a replica does not implement, the output lists it:

Output

running locally (worlds in a server this command starts)
run_started runId=run_20261031-234925_79b4774ed98fa32d scenes=["starter"] trials=1 parallel=1 workspace=local:/path/to/project target=local
trial_started scene=starter trial=1 envId=2uvxqv90
trial_finished scene=starter trial=1 passed=true valid=true warnings=[] exit=exited exitCode=0 realMs=73 failedChecks=[]

✓ starter  1/1 (100%)
    coverage gap: POST /api/chat.scheduleMessage (slack) x1

agent not sealed: network open; environment limited to 9 variables from this shell (pass more with --pass-env NAME)
pass (minimum pass rate 100%)  run run_20261031-234925_79b4774ed98fa32d
next: ancha baseline save before

The call gets an error in the tool’s own shape, unknown_method for Slack, so the agent sees a failure. Without --strict, a gap does not change the exit code. With --strict, any gap exits 4 and the output adds --strict: the agent hit an endpoint Ancha does not implement (see coverage gaps). ancha explain lists the gaps either way. Check ancha twins coverage <tool> for what each tool implements.

Where the agent runs

--on picks where the agent itself runs; the default is what ancha.yaml says.

`--on`	The agent runs
`local`	As a process on this computer, with the environment described above.
`docker`	In the sealed container `agent.execution` describes, which needs an `image`. Never a fallback to a process. See ancha.yaml.
`cloud`	In your hosted workspace: `ancha run --on cloud` uploads this project, registers its agent under the project folder’s name (or updates it), connects it, and runs each test there as its own run, streaming the progress. The exit code is the worst of the runs. Running your own agent in the workspace is in private beta; until your workspace has access, the command is refused with `hosted runs of your own agent are in private beta for this workspace`. See Hosted workspace.

--on cloud and --upload do not combine: a run in the workspace already lives there.

Results in your workspace

--upload, or ANCHA_UPLOAD=1 in the environment, sends the finished run to your workspace, where the console shows it at https://app.ancha.io/runs/<run id>. It needs a workspace: ancha login, or ANCHA_SERVER_URL and ANCHA_API_KEY. The run’s own files stay in .ancha/runs/.

Output

pass (minimum pass rate 100%)  run run_20261031-234205_e9f7049f942a39d5
uploaded: https://app.ancha.io/runs/run_20261031-234205_e9f7049f942a39d5

A failed upload is a warning: the run keeps its own exit code, and ancha runs upload <run id> tries again. --upload-required makes a failed upload fail the command instead. A run started with --detach uploads when it finishes, and ancha wait and ancha status say where the results went. With --json, run_finished carries upload: url, bytes and files, replaced: true when the run was uploaded before, or error. See Results in your workspace for what the upload holds and what is redacted.

Retrying safely

A caller whose command can time out, such as a coding agent or a CI step, passes --idempotency-key <key> (1 to 200 visible ASCII characters). Running the same command again with the same key returns the original run instead of starting another: its summary and exit code, with replayed: true in the JSON. While the original is still running, the retry fails with SESSION_CONFLICT and a hint to ancha wait <id>. The same key with a different scene set, agent or options is a usage error:

Output

error[USAGE]: this --idempotency-key was used for a different run
hint: use a new key for a new run; reuse a key only to retry the same command

Detached runs

--detach hands the run to a background worker and returns at once:

ancha run -n 3 --detach --json

Output

{"event":"run_detached","runId":"run_20261031-234928_4ef35808ffd351f9"}

Follow it with ancha logs <id> --follow, ancha wait or ancha status. logs --follow and wait exit with the run’s own exit code. Pass the same --out to all of them if you changed it. A detached run survives the launching shell, but not a reboot or a killed worker.

Timeouts and failures

An agent that runs past the scene’s timeout is stopped and the trial records exit=timeout. An agent that exits non-zero records its exit code. In both cases the checks still run against what the agent did, and ancha explain shows the end of its log. Ctrl-C stops the run the same way and keeps the evidence collected so far.

Model-graded checks

Judge checks need OPENROUTER_API_KEY in the environment of ancha run. Without it they are skipped, and a skipped required check fails its trial. The key is read by ancha run itself; the agent does not get it unless you list it in agent.pass_env. See Checks.

Your own harness

To start the agent yourself and still grade it with the scene’s checks, start a world with ancha world up, run the agent against it, and grade it with ancha check. See World lifecycle.

Get started

Guides

Concepts

Tools

CLI

Reference

Cloud

Help

Example

How a run works

What the agent gets

Exit codes

JSON output

Compare with a baseline

Coverage gaps and `--strict`

Where the agent runs

Results in your workspace

Retrying safely

Detached runs

Timeouts and failures

Model-graded checks

Your own harness

​Example

​How a run works

​What the agent gets

​Exit codes

​JSON output

​Compare with a baseline

​Coverage gaps and --strict

​Where the agent runs

​Results in your workspace

​Retrying safely

​Detached runs

​Timeouts and failures

​Model-graded checks

​Your own harness

Example

How a run works

What the agent gets

Exit codes

JSON output

Compare with a baseline

Coverage gaps and `--strict`

Where the agent runs

Results in your workspace

Retrying safely

Detached runs

Timeouts and failures

Model-graded checks

Your own harness