Test an app

​

Configure ancha.yaml

agent:
  start: ./ancha/start.sh
  url: "http://127.0.0.1:{{agent.port}}/api/slack/events"
  ready_url: "http://127.0.0.1:{{agent.port}}/api/health"
  startup_timeout: 180s
  idle: 20s
  setup_calls: true
  clock: node
  env:
    APP_SLACK_TOKEN: "{{slack.token}}"
scenes: [ancha/scenes/*.yaml]
trials: 3
parallel: 1

​

Write the start script

• Set database and cache URLs and the app’s test secrets in the script.
• For a real key, such as the app’s model key, add its name to agent.pass_env in ancha.yaml, or run with --pass-env NAME. Its value stays in your shell and out of the script.

1. Restore the database from a template copy: CREATE DATABASE app_trial TEMPLATE app_base, with app_base migrated once.
2. Flush the caches and queues the tests own.
3. Seed what the flow needs with the app’s own seeds, factories or test helpers. Store the world’s token where the app keeps its OAuth tokens.
4. exec the server. Start workers, such as job queues, from the same script. Ancha stops the whole process group after the attempt.

• Old rows, caches and sync cursors, such as a stored history ID or a last-seen timestamp, make an app silently do nothing in a new world.
• A world’s IDs repeat from attempt to attempt, as its seed does. Clear the app’s dedupe keys and locks at the start of every attempt. Run attempts one at a time (parallel: 1) unless each attempt gets its own database and cache namespace.
• Seed what a real account has, through the app’s own functions: the rules and settings its onboarding or settings page creates. A test helper’s shortcut, such as a missing default rule or a rule without the ID the settings page would store, changes what the app does.
• A production build boots faster than a development server, and boot time counts against startup_timeout. .ancha/runs/<run>/trial-*/agent.log has the script’s and the app’s output.

​

Point the app at the tools

• Use the app’s base-URL setting if it has one. ancha connect <tool> --lang ts or --lang py prints the SDK settings.
• If the SDK has no such setting, add a development-only initializer that reads the tool’s variable and does nothing when it is unset. Keep app changes that small and say why in a comment.
• Google clients keep only the host of a rootUrl set on the client. Ancha routes those calls by their token.
• Gmail reaches an app the way it does in production: the app calls users.watch, and the world then pushes Pub/Sub messages to agent.url. If the app registers its watch on a schedule, call that route with an app_calls entry early in the scene. The world gives the mailbox’s address as GMAIL_ADDRESS, and GET /gmail/v1/users/me/profile returns the same address. See Gmail.
• Identifiers a scene relies on, such as an inbox token, a cron secret or an API key the world sends, should be fixed values your seed creates, because a scene is static.

​

Write scenes for the app

id: stale-question
title: A customer's question gets an answer in the thread, and a reminder two days later if nobody resolved it
twins: [slack]
start_time: 2026-11-02T15:00:00Z
timeout: 3d
skip_idle: true
seed:
  slack:
    users:
      - { ref: bot, name: support-bot, is_bot: true }
      - { ref: dan, name: dan, real_name: Dan Ito }
    channels:
      - { ref: support, name: support, members: [$bot, $dan] }
world_rules:
  actors:
    - at: 5s
      slack.post: { ref: question, channel: $support, user: $dan, text: "How do I export my invoices?" }
  app_calls:
    - at: 2d1h
      path: /api/cron/stale-questions
      headers: { authorization: Bearer test-cron-secret }
checks:
  - id: answered_in_thread
    state: { slack.message: { channel: $support, from: $bot, in_thread_of: $question } }
  - id: no_deletes
    trace: { not_called: "* /api/chat.delete" }

• start_time on a Monday suits apps that count business days. skip_idle: true jumps the clock while the app and the world are quiet.
• world_rules.actors act in a tool at a time, for example slack.post or gmail.send. See World rules.
• world_rules.app_calls call the app: its cron routes, or a customer message into its API. Each takes at, method (default GET), path, headers and body, and goes to the origin of agent.ready_url.
• ancha twins seed <tool> --json prints each tool’s seed fields, checks and actors.

​

People who answer

people:
  - ref: tom
    name: Tom
    role: a client at another company who answers email himself, briefly, within a day or two
    accounts: { gmail: tom@client.example }
    replies_after: 1d..2d

​

Make every check able to fail

​

Turn a production trace into a test

1. ancha import --offline <trace file> reads an Ancha Trace IR file, or an OTLP JSON export whose spans carry the bodies of the app’s calls to its tools. It writes scenes/from-traces/trace-<id>/scene.yaml and provenance.json and uploads nothing.
2. It lists the questions one trace can’t settle, each with the default the test uses. A default repeats what production did, so answer the ones about what went wrong: add --answer <id>=<value> for each and import again with --force.
3. Run the scene against the build that failed in production. It should fail the way production did. Then fix the app and run again.

• A trace holds the app’s calls, not what started each one. The import lists the calls production made before the first event the test replays: the app’s own setup, such as a watch or its labels. The test has them only if the app makes them when it starts, or the start script or an app_calls entry does.
• When a test doesn’t fail the way production did, ancha traces diff <trace file> <run>/<trial> lists the calls only production made, the ones only the attempt made, and the first call where their orders part. A call production made and the attempt didn’t, such as labels created at setup or a settings read, usually points at what the seed lacks.