> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ancha.io/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> These docs describe the Ancha CLI, npm package @ancha-labs/cli, and the hosted workspace at app.ancha.io. Coding agents should start at https://docs.ancha.io/coding-agents. Run the CLI as `ancha` after `npm install -g @ancha-labs/cli`, or as `npx -y @ancha-labs/cli` without installing. Pass --json for machine-readable output, or set ANCHA_JSON=1. Every command ends with a `next:` line naming the next command. Exit codes: 0 pass, 1 a check failed or a comparison regressed, 2 usage, config or authentication error, 3 infrastructure error, 4 unimplemented endpoint with --strict.

# From a production trace to a failing test

> Turn the production run that went wrong into a test on your computer: the agent's own tools answer as they did in production, the test checks what went wrong, and it tells you whether the failure comes back.

Your agent did something wrong in production. A customer complained, or you saw it in your tracing tool. This guide turns that run into a test you run on your computer. Your agent runs for real, on the same input. Each of its tool calls gets the answer production got. The test checks for what went wrong and tells you whether the failure happens again. The test stays in your repository, so once you fix the agent the failure cannot quietly come back.

## Before you start

* The Ancha CLI: `npm install -g @ancha-labs/cli`.
* An agent whose runs are traced: OpenTelemetry (the AI SDK, OpenLLMetry, any GenAI instrumentation), Langfuse or LangSmith.
* For traces kept in your workspace: an account at [app.ancha.io](https://app.ancha.io/login), then `ancha login`. To keep everything on your computer, use `ancha import --offline` with a trace file instead.

## 1. Wrap your agent's tools

In a test, your agent's tools must not reach production. Wrap them once. Outside a test the wrapper calls your function as before. In a test, each call is answered from production's recorded calls.

```ts TypeScript theme={null}
import { wrapTools, entry } from '@ancha-labs/tools';

const tools = wrapTools({ lookupOrder, refundOrder }); // functions, or the AI SDK's tool() objects

await entry(async input => runAgent(input.task, tools)); // the command a test runs
```

```python Python theme={null}
import ancha

@ancha.tool
def lookup_order(order_id: str) -> dict: ...

tools = ancha.wrap_tools([search_tool, update_tool])  # LangChain tools, OpenAI Agents function tools

ancha.entry(lambda input: run_agent(input["task"]))  # the command a test runs
```

Install with `npm install @ancha-labs/tools` or `pip install ancha`. Tools your agent reaches over MCP need no wrapper: point the MCP client at `{{tools.mcp_url}}` in `ancha.yaml`. [Tool wrappers](https://docs.ancha.io/reference/tool-wrappers.md) has the details.

If a tool the trace recorded is not wrapped, the test stops before your agent runs and names the tool, so a test never calls production by accident.

## 2. Bring in the trace

Pick the source your agent already uses.

> [tab] Langfuse or LangSmith
>
> Import one trace by its id, with your own keys from this shell. They never leave your computer.
>
> ```bash theme={null}
> export LANGFUSE_PUBLIC_KEY=... LANGFUSE_SECRET_KEY=... LANGFUSE_HOST=https://cloud.langfuse.com
> ancha import --from langfuse 6ab868e05ac8c1e5910fdfc70d565af0 --expect "it refunded an order that belongs to another customer"
> ```
>
> For LangSmith, set `LANGSMITH_API_KEY` and pass `--from langsmith`. Ancha reads the whole conversation of a thread and the feedback on every turn.

> [tab] OpenTelemetry
>
> On the **Traces** page of your workspace, click **Create ingest key**, and copy the two lines it shows into your agent's production environment:
>
> ```bash theme={null}
> OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://ingest.ancha.io/v1/traces
> OTEL_EXPORTER_OTLP_TRACES_HEADERS=Authorization=Bearer%20<your ingest key>
> ```
>
> An ingest key can only send traces, so it is safe in production configuration. Each trace is ready about a minute after its last span. Find the run and pull its test:
>
> ```bash theme={null}
> ancha traces --failed
> ancha pull fc7f4f10 --expect "it declared the incident resolved on a status measured before the rollback"
> ```

> [tab] A trace file
>
> ```bash theme={null}
> ancha import --offline trace.otlp.jsonl --expect "it refunded an order that belongs to another customer"
> ```
>
> `--offline` makes the test on this computer and uploads nothing. The file is OTLP JSON, a Langfuse or LangSmith export, or [Ancha Trace IR](https://docs.ancha.io/reference/trace-ir.md).

`--expect` says in one sentence what went wrong. The test gets a check for it, and Ancha judges whether production did it (it did). Without `--expect`, the checks come from the trace: a thumbs-down comment becomes a check that the agent does not do what the user complained about, and an error that ended the run becomes a check that the agent finishes. With neither, the test asks what the agent should have done.

The test lands in `scenes/from-traces/trace-<id>/`:

| File | What it holds |
| - | - |
| `scene.yaml` | What the agent is asked, its tools, and the checks. |
| `observations.jsonl` | Every tool call production made, with its answer. |
| `production.jsonl` | Production's run: its input, its tool calls and its answer. Each check's production result comes from it. |
| `questions.json` | What one trace cannot settle, each with the default the test uses. |

## 3. Answer what one trace cannot settle

The import lists the questions. The two you meet most:

* `tools.mcp`: which tools your agent reaches over MCP. A trace cannot tell. The test serves those at `{{tools.mcp_url}}`.
* `tools.simulate_writes`: which writes may be simulated when the agent makes one production never made. By default such a write fails, so a test never invents a write's success.

```bash theme={null}
ancha import --offline trace.otlp.jsonl --force --answer tools.mcp=get_deploy_status,rollback --answer tools.simulate_writes=post_status_update
```

For a trace in your workspace, answer with `ancha pull <id> --answer ...`, which saves a new version. See [ancha pull](https://docs.ancha.io/cli/pull.md#answers).

## 4. Reproduce it

Run the test against the version of the agent that failed in production:

```bash theme={null}
ancha run scenes/from-traces/trace-3d4e261c/scene.yaml --trials 3
```

```text Output theme={null}
computed production's result for what_went_wrong, what_the_user_complained_about from scenes/from-traces/trace-3d4e261c/production.jsonl
✗ trace-3d4e261c  0/3 (0%)
  no-deletes                      passed in production · failed 0/3 here
  what_went_wrong                 failed in production · failed 3/3 here
  what_the_user_complained_about  failed in production · failed 3/3 here
=> production's failure happens here too
   answers to the agent's tools: 9 replayed
```

The last line says how much of the run stood on production's own answers. A call production never made to a tool that only reads is answered by a model shown the tool's description, every answer production got, and the agent's writes so far. A tool that changed something is never simulated unless you said so. `ancha explain` lists each answer that was not production's.

If production's failure does not happen here, either the agent is already fixed or the test differs from production. `questions.json` says where it may differ.

## 5. Fix the agent and keep the test

Change your agent and run the test again until production's failure no longer happens:

```text Output theme={null}
✓ trace-3d4e261c  3/3 (100%)
  no-deletes                      passed in production · failed 0/3 here
  what_went_wrong                 failed in production · failed 0/3 here
  what_the_user_complained_about  failed in production · failed 0/3 here
=> production's failure did not happen here: the agent is fixed, or the world differs from production (see questions.json next to the scene)
   answers to the agent's tools: 3 replayed
```

Run it several times. An agent that passes once can still fail. Then commit `scenes/from-traces/`. Every `ancha run`, in CI too, keeps the failure fixed. See [Run Ancha in CI](https://docs.ancha.io/cloud/ci.md).

Add checks of your own in `scene.yaml`, such as `refund_order {order_id: A-1002}` never being called. Ancha computes production's result for them too, and they stay when you pull a newer version. See [Checks](https://docs.ancha.io/concepts/checks.md).

## When the agent calls vendor APIs directly

An agent that calls Slack, GitHub, Datadog or another tool Ancha has a twin of, with the request and response bodies in its trace, gets that twin seeded from the calls. The test then answers new calls from the twin's state, not from a recording. Add the bodies with the [Ancha recorder](https://docs.ancha.io/reference/otlp.md#request-and-response-bodies). A tool of your own whose code called such a vendor runs against the twin for calls production never made.

## What a trace can and cannot tell

* It shows what your agent was asked and what each tool answered. It cannot show what a tool would have answered to a call production never made; that is replayed from a similar call, simulated, or failed, and each answer says which.
* It cannot test a bug inside a tool's own code. Ancha answers in the tool's place; what is tested is what your agent decides.
* Your trace data stays in your workspace, redacted before it is stored. See [Trace privacy](https://docs.ancha.io/cloud/trace-privacy.md).


## Related topics

- [Traces](https://docs.ancha.io/cloud/traces.md)
- [ancha traces](https://docs.ancha.io/cli/traces.md)
- [ancha import](https://docs.ancha.io/cli/import.md)
- [ancha pull](https://docs.ancha.io/cli/pull.md)
- [Changelog](https://docs.ancha.io/changelog.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.