For coding agents

This page tells a coding agent such as Claude Code, Codex or Cursor how to test and fix an AI agent with the Ancha CLI, version 0.1.4. The same text is at /coding-agents.md as Markdown, and the whole site is at /llms-full.txt.

Install

Ancha needs Node.js 24 or later.

Option	Command	Then run Ancha as
Global install	`npm install -g @ancha-labs/cli`	`ancha`
Project dev dependency	`npm install --save-dev @ancha-labs/cli`	`npx @ancha-labs/cli`
No install	nothing	`npx -y @ancha-labs/cli@0.1.4`

Check the install with ancha --version. It prints 0.1.4.

With npx, use the full package name, @ancha-labs/cli. There is no ancha package on npm. The instruction files ancha init writes say the same: run every command as npx -y @ancha-labs/cli <command> until the CLI is installed, then as ancha <command>.

The three commands

Run every command from the project root, where ancha.yaml lives.

Step	Command	What it does
1. Set up	`ancha init`	Writes `ancha.yaml` with the start command it finds in `package.json`, a Procfile, `pyproject.toml` or the Dockerfile, a starter scene for the tools your dependencies use, agent instructions and `.ancha/` in `.gitignore`. Installs nothing, runs nothing. Pass `--agent "<command>"` when nothing names a command.
2. Run	`ancha run --json`	Runs the agent against every scene in `ancha.yaml`. Each trial gets a fresh copy of the tools.
3. Explain	`ancha explain latest --json`	For each failed check: expected, actual, and the API calls behind it. Also crashes and timeouts with the end of the agent’s log, coverage gaps, and what the agent’s environment withheld.

ancha doctor --json checks the setup between steps 1 and 2. It runs the agent once and confirms it reached each tool. A tool the agent did not call is a warning, not a failure, and doctor exits 0. Every command ends with next: <exact command> in text, and with a next list in JSON. Follow it. To prove a fix, save the failing run, change the agent, and compare:

ancha baseline save before
# edit the agent's code
ancha run --compare before --json

Exit code 0 means every check passed and no scene and no check got worse than before.

How your agent reaches the tools

Ancha starts the agent with tool URLs and fake credentials in environment variables, for example SLACK_API_URL and SLACK_BOT_TOKEN. The agent must use them instead of real service URLs. ancha connect <tool> --lang ts (or --lang py) prints SDK settings for a tool. See Environment variables for the full list. The agent runs as a process on this computer with this computer’s network. Its environment holds a base set (PATH, HOME, locale and the like), Ancha’s variables and the tools’ variables. Anything else from the shell, such as a provider key, reaches it only through agent.pass_env in ancha.yaml or --pass-env NAME. For a sealed run, set agent.execution: { mode: container, image: node:24-slim } in ancha.yaml; it needs Docker. See ancha run.

JSON output

Add --json to any command for machine-readable output on stdout, or set ANCHA_JSON=1, which acts as --json on every command.

ancha run --json, ancha wait --json and ancha logs --json print newline-delimited JSON (NDJSON): one event per line, ending with run_finished. Do not parse stdout as a single JSON object.
Every other command prints one JSON object.
Errors print one object with an error key, and the process exits with the code in the table below.
ancha schema <kind> --json prints the JSON Schema of ancha.yaml, scene files and these outputs: run_started, run_finished, explain, world_up, check and error.

Error

{"error":{"code":"USAGE","message":"unknown run \"run_nope\"","hint":"see .ancha/runs/"}}

Events from ancha run --json, in order:

Event	Fields
`run_started`	`runId`, `scenes`, `trials`, `parallel`, `workspace` (the world server, or `local:<project path>`), `target` (`local` or `docker`)
`trial_started`	`scene`, `trial`, `envId`
`trial_finished`	`scene`, `trial`, `passed`, `valid`, `warnings`, `exit`, `exitCode`, `realMs`, `failedChecks`
`run_finished`	`summary`: the full result, described below; `next`: the commands to run next; `replayed`: `true` when an idempotency key returned an earlier run

ancha run --json (failing run)

{"event":"run_started","runId":"run_20261031-234207_17fd906ac2fe6ce5","scenes":["starter"],"trials":1,"parallel":1,"workspace":"local:/path/to/project","target":"local"}
{"event":"trial_started","scene":"starter","trial":1,"envId":"vfbwwx5i"}
{"event":"trial_finished","scene":"starter","trial":1,"passed":false,"valid":true,"warnings":[],"exit":"exited","exitCode":0,"realMs":76,"failedChecks":["github_access"]}
{"event":"run_finished","summary":{"runId":"run_20261031-234207_17fd906ac2fe6ce5","startedAt":"2026-10-31T23:42:07.666Z","finishedAt":"2026-10-31T23:42:07.755Z","agent":"node agent.mjs","minPassRate":1,"verdict":"fail","scenes":[{"sceneId":"starter","trials":1,"passed":0,"passRate":0,"passAllK":false,"checks":{"slack_access":{"passed":1,"total":1,"required":true},"github_access":{"passed":0,"total":1,"required":true},"datadog_access":{"passed":1,"total":1,"required":true},"pagerduty_access":{"passed":1,"total":1,"required":true}},"agentIssues":[],"harnessIssues":[],"invalidTrials":0,"skippedJudges":0,"warnings":[],"failures":[{"sceneId":"starter","trial":1,"check":"github_access","expected":"agent calls * /*repos*","actual":"never called; agent made 3 call(s)","explanation":"The agent never made a call matching \"* /*repos*\". Its calls were: GET /api/conversations.list, GET /api/v1/dashboard, GET /services. Patterns are \"<METHOD> <PATH>\" matched against the service-relative path, and * is a wildcard.","traceExcerpt":[{"seq":1,"source":"agent","method":"GET","path":"/api/conversations.list","status":200},{"seq":2,"source":"agent","method":"GET","path":"/api/v1/dashboard","status":200},{"seq":3,"source":"agent","method":"GET","path":"/services","status":200}]}],"coverageGaps":[],"meanRealMs":76,"agentCalls":3,"trialsDetail":[{"trial":1,"passed":false,"valid":true,"exit":"exited","exitCode":0,"realMs":76,"simMs":74,"sessionId":"226d6522146dc2ce4fffc2f6f79d3412","dir":"/path/to/project/.ancha/runs/run_20261031-234207_17fd906ac2fe6ce5/trial-starter-1"}]}],"sceneFingerprints":{"starter":"9502d255c5e452600216c2fba59d25116a5db5694e0678b5ed3122237f9cac3f"},"agentEnvironment":{"target":"host","inherited":9,"withheld":72},"workspace":"local:/path/to/project","target":"local"},"next":["ancha explain run_20261031-234207_17fd906ac2fe6ce5"]}

Retrying safely

If your command can time out, give ancha run an --idempotency-key <key>. Running it again with the same key returns the same run instead of starting another, with replayed: true. While the first run is still going, the retry fails with SESSION_CONFLICT and a hint to ancha wait <id>. Use a new key for a new run. ancha run --detach --json returns a run ID at once; follow it with ancha logs <id> --follow --json, which exits with the run’s exit code.

Exit codes

Code	Meaning	What to do
`0`	Every required check passed, the pass rate met `min_pass_rate`, and nothing regressed.	Done.
`1`	A check failed, the pass rate was too low, or a comparison regressed. A single check’s drop counts; see `comparison.droppedChecks`.	Run `ancha explain latest --json` and fix the agent.
`2`	Usage, config, scene or authentication error. `AUTH_INVALID` means `ancha login` again or a valid `ANCHA_API_KEY`.	Read the `hint`. Fix the command or the file named in the error.
`3`	Infrastructure error: the server, the agent process or a trial’s harness failed.	Not an agent failure. Run again, and `ancha doctor --json` if it repeats.
`4`	The agent called an endpoint the replica does not implement. Only with `--strict`.	`ancha explain` lists the gaps either way. Check `ancha twins coverage <tool>` before changing the agent.

When several apply, 3 wins over 4, and 4 wins over 1. ancha explain exits 0 even when it lists failures, so read the exit code of ancha run.

Read a verdict

The run_finished summary holds the verdict. These fields matter most:

Field	Meaning
`verdict`	`pass` or `fail` for the whole run.
`workspace`, `target`	Where the worlds ran, and whether the agent ran as a local process or in a container.
`scenes[].passed`, `scenes[].trials`, `scenes[].passRate`	Trials that passed out of trials run. A trial passes when every required check passes.
`scenes[].checks`	Per check: `passed`, `total`, `required`.
`scenes[].failures`	Failed checks with `expected`, `actual`, `explanation` and `traceExcerpt`.
`scenes[].agentIssues`	The agent crashed, exited non-zero or timed out.
`scenes[].harnessIssues`, `scenes[].invalidTrials`	Ancha itself failed during a trial. These trials are not agent failures.
`scenes[].coverageGaps`	Calls to endpoints the replica does not implement, with counts.
`scenes[].trialsDetail[].dir`	The trial’s folder, with `agent.log`, `trace.jsonl`, `state.json` and `checks.json`.
`agentEnvironment`	`target`, and how many shell variables the agent `inherited` and how many were `withheld`.
`comparison.regressed`, `comparison.droppedChecks`	With `--compare`: whether anything got worse, and which checks dropped.

Read it in this order:

Exit 2 or 3: fix the setup or retry. Do not change the agent yet.
agentIssues is not empty: the agent crashed or timed out. ancha explain shows the end of agent.log.
failures is not empty: compare expected with actual, then read the trace. Many bugs only appear with the world’s data, so read the evidence before guessing from the code.
coverageGaps is not empty: the replica answered with an error because it does not implement that call. Check ancha twins coverage <tool> before you change working code.
agentEnvironment.withheld is not zero and the agent needs one of those variables, such as a provider key: pass it with --pass-env NAME or agent.pass_env.

A clean process exit does not mean the task passed. Only checks decide that.

Rules for coding agents

Fix the agent’s code. Do not edit ancha.yaml, the scenes or their checks to make a run pass. The exception is a check that is wrong for the real API, such as a method the official SDK never sends: tell the user, show the evidence from ancha explain and the vendor’s documentation, and change it only with their agreement.
Never point the agent at real SaaS URLs, and never give it real credentials.
Never give the agent under test the scenes, checks, seeds or reference companions (oracle.mjs, null.mjs).
Use --compare against a saved baseline to prove a fix. Never compare a run with itself.
Keep --json on every command, and read the exit code.

The starter scene only proves the agent can reach each tool. Replace it with a scene that checks real work. Its checks accept any HTTP method (* /api/*.* for Slack), so an agent built on an official SDK passes them as it is.

Hosted workspace

If the user has an Ancha workspace, ancha login asks them to approve this computer once in the browser; ask them to do it. A signed-in ancha run still runs the agent and its worlds on this computer, and its first line names the workspace. ancha run --upload sends the finished run to that workspace, where the console shows it. ANCHA_SERVER_URL, server.url in ancha.yaml or --server-url put the worlds in the workspace instead, and --local keeps a command on this computer. ancha whoami --json shows which workspace commands use. In CI, ANCHA_SERVER_URL and ANCHA_API_KEY replace the login, and ancha run --local --upload keeps the worlds on the runner and sends the results to the workspace. See Hosted workspace and Results in your workspace.

Your own harness

To start the agent yourself, start a world instead of a run:

ancha world up --scene scenes/support-reply/scene.yaml --json   # envId, and env: the tools' variables and the task
# run the agent with exactly the variables in env
ancha check --env <envId> --json                                 # the scene's checks; exit 1 when a required one fails
ancha world down --env <envId>

ancha world trace --env <envId> --json shows the calls so far. See World lifecycle. ancha world gen "<description>" --json writes a world file from a description; it needs OPENROUTER_API_KEY. See ancha world gen.

What `ancha init` writes for coding agents

File	Contents
`AGENTS.md`	A managed section between `<!-- ancha:begin -->` and `<!-- ancha:end -->` with the loop and the rules. Existing content is kept.
`.claude/skills/ancha/SKILL.md`	A Claude Code skill named `ancha` with the same loop, the tools’ variables and the exit codes.

See ancha init.

Prompt for your coding agent

Paste this into Claude Code, Codex, Cursor or another coding agent, in the repo of the agent you want to test:

Prompt

Use Ancha to test the AI agent in this repository and fix what fails.
Docs for you: https://docs.ancha.io/coding-agents.md

1. If `ancha --version` fails, run `npm install -g @ancha-labs/cli`.
2. If there is no ancha.yaml, run `ancha init` (add --agent "<command that starts the agent>" if it finds none), then `ancha doctor --json`.
   Make the agent read tool URLs and credentials from the environment variables Ancha sets.
   `ancha connect <tool> --lang ts` (or `--lang py`) prints the SDK settings.
   If the agent needs a provider key, list its name under agent.pass_env in ancha.yaml.
3. Run `ancha run --json`. Exit 0 means every check passed.
   Exit 1 means a check failed: run `ancha explain latest --json` and read expected, actual and the trace before you change code.
4. Before fixing, save the failing run: `ancha baseline save before`.
   After fixing, prove it: `ancha run --compare before --json`. Stop when it exits 0.
5. Fix the agent's code, not ancha.yaml, the scenes or their checks.
   Exit 2 is a setup or authentication error, 3 is infrastructure, 4 means the agent called an endpoint Ancha does not implement.
   Report those to me instead of working around them.
6. Never use real SaaS URLs or credentials. Never give the agent under test the scenes, checks or reference companions.

Finish with what failed, what you changed, and the final `ancha run --compare before --json` result.

Discover the CLI

ancha --help is one page; ancha advanced lists the rest; ancha --help --json prints every command and flag as one JSON tree.
ancha <command> --help prints one command’s flags and defaults.
ancha schema --json prints every schema; ancha docs <topic> prints the bundled guide.
CLI reference documents every command in 0.1.4.
Errors lists every error code.

Get started

Guides

Concepts

Tools

CLI

Reference

Cloud

Help

For coding agents

Install

The three commands

How your agent reaches the tools

JSON output

Retrying safely

Exit codes

Read a verdict

Rules for coding agents

Hosted workspace

Your own harness

What `ancha init` writes for coding agents

Prompt for your coding agent

Discover the CLI

​Install

​The three commands

​How your agent reaches the tools

​JSON output

​Retrying safely

​Exit codes

​Read a verdict

​Rules for coding agents

​Hosted workspace

​Your own harness

​What ancha init writes for coding agents

​Prompt for your coding agent

​Discover the CLI

Install

The three commands

How your agent reaches the tools

JSON output

Retrying safely

Exit codes

Read a verdict

Rules for coding agents

Hosted workspace

Your own harness

What `ancha init` writes for coding agents

Prompt for your coding agent

Discover the CLI