Skip to main content
The pytest-ancha plugin gives each test a world fixture and records each executed test as a local run. Python code uses import ancha.

​
Install the plugin

The package is not published on PyPI. Install a supplied wheel in the Python environment where you run pytest:
The plugin loads automatically. Use Ancha CLI 0.1.5 or later for world capture and world record. ANCHA_CLI can name another CLI command.

​
Write a test

Use a scene file containing the tools, starter data and checks for your agent. Run pytest from the project directory:
world.start gives the process the world’s tool URLs and fake credentials. Your agent runs on your computer. wait_until_quiet waits for that process to exit; its timeout is in real seconds. The fixture stops any remaining process, records a world observation and removes the world after the test. The marker accepts scene, test (an alias for scene), world (a saved world ID), on, seed and ttl. Without a scene or saved world, the fixture uses the single scene named in ancha.yaml. world.check() returns a report. A failed check makes the report false. Invalid checks, unknown references and CLI failures raise ancha.AnchaError, with code, message and optional hint. A false check does not fail pytest until you assert it:
This assertion passes when the agent did not call chat.postMessage. The recorded run preserves the observed check without turning it into a required passing check.

​
Choose where the world runs

--on cloud uses the hosted workspace selected by the CLI. Sign in with ancha login first, or configure a server URL and API key. The world runs there; pytest and the agent stay on your computer. Run files also stay on your computer. The pytest --on option overrides the marker’s on. Without it, the marker wins. With neither, the CLI’s normal world selection applies: login alone keeps worlds local; an explicit server URL can select another server. --on local forces local worlds.

​
Repeat attempts

Each attempt starts with a fresh world. A failed assertion counts as a failed attempt. The test passes when its pass rate meets the threshold. Setup, capture and cleanup errors remain infrastructure failures; they do not count as failed agent answers.

​
Read the saved run

Each executed pytest item produces a run under .ancha/runs/, with source: pytest and target: local. Parameterized cases are separate runs; parameter values are replaced by a hash in the recorded test ID. Tests without a world also get a record. --collect-only creates none.
The record contains pytest phase outcomes, each attempt and its available world state, trace and check observations. Pytest assertions determine the result. Skips and expected failures stay ungraded. A world capture or cleanup failure makes the affected attempt invalid. The record preserves a passing call followed by a failing teardown as separate phase results. The plugin does not save environment values, raw exception messages, captured test output or agent logs. Keep .ancha/ out of version control. Uploading pytest records is not supported; they remain local.