CI - Ancha

ancha run exits 0 only when every scene meets its pass rate and no check regressed, so it works as a CI gate as is. You can run it two ways.

Mode	Needs	Where the tools run
Local	Node.js 24 in the CI job	A server `ancha run` starts inside the job. The results stay in the job.
Results in your workspace	`ANCHA_SERVER_URL` and `ANCHA_API_KEY` secrets, and `--upload`	A server inside the job with `--local`; the finished run is sent to your hosted workspace, where the console shows it. Leave out `--local` to run the worlds in the workspace, which uses its machine time. See Results in your workspace.

In both modes the agent runs inside the CI job, so its own dependencies must be installed there. The agent does not see the job’s environment: it gets a base set, Ancha’s variables and the tools’ variables. Name the keys it needs, such as a model key, in agent.pass_env in ancha.yaml or with --pass-env. ANCHA_API_KEY stays with the CLI.

GitHub Actions

.github/workflows/ancha.yml

name: Agent tests
on: [push, pull_request]

jobs:
  ancha:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 24
      - run: npm ci
      - name: Run Ancha
        run: npx -y @ancha-labs/cli@0.1.4 run --local --upload --json --pass-env OPENROUTER_API_KEY
        env:
          # Results in your workspace. Leave these two out, and --upload, to keep everything inside the job.
          ANCHA_SERVER_URL: ${{ secrets.ANCHA_SERVER_URL }}
          ANCHA_API_KEY: ${{ secrets.ANCHA_API_KEY }}
          # Keys your agent itself needs, if any. --pass-env hands them to the agent.
          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
      - name: Explain failures
        if: failure()
        run: npx -y @ancha-labs/cli@0.1.4 explain latest

Pin the CLI version so a new release cannot change your results unannounced. Get ANCHA_SERVER_URL and a key from Settings; see Login and API keys. The first line of the run’s output says where its worlds ran, and the last line where the results went. A failed upload is a warning; add --upload-required to fail the job without its results, or run ancha runs upload latest as a later step. A job step that can time out can pass --idempotency-key with a value unique to the commit; a retry with the same key returns the original run instead of starting another. See ancha run.

Gating

Exit code	Meaning in CI
`0`	Pass.
`1`	The agent failed checks or regressed. Fail the build.
`2`	A config, scene or authentication error. Fix the repository or the secrets.
`3`	Infrastructure failed. Retry the job.
`4`	With `--strict`, the agent used an unimplemented endpoint.

Agents that use a model are not deterministic. Run several trials and set a pass-rate floor instead of demanding every trial pass:

ancha.yaml

trials: 5
min_pass_rate: 0.8

Judge checks in CI

Judge checks need OPENROUTER_API_KEY in the job’s environment. ancha run reads it itself; the agent only gets it through --pass-env or agent.pass_env. Without the key, judge checks are skipped, and a required skipped check fails its trial. Keep rubrics you are still tuning out of pass. See Checks.

Get started

Guides

Concepts

Tools

CLI

Reference

Cloud

Help

CI

GitHub Actions

Gating

Judge checks in CI

​GitHub Actions

​Gating

​Judge checks in CI

GitHub Actions

Gating

Judge checks in CI