> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ancha.io/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> These docs describe the Ancha CLI, npm package @ancha-labs/cli version 0.1.4, and the hosted workspace at app.ancha.io. Coding agents should start at https://docs.ancha.io/coding-agents. Run the CLI as `ancha` after `npm install -g @ancha-labs/cli`, or as `npx -y @ancha-labs/cli@0.1.4` without installing. Pass --json for machine-readable output, or set ANCHA_JSON=1. Every command ends with a `next:` line naming the next command. Exit codes: 0 pass, 1 a check failed or a comparison regressed, 2 usage, config or authentication error, 3 infrastructure error, 4 unimplemented endpoint with --strict.

# Test a Slack bot

> Write a scene for a Slack support bot built on the official SDK, let Ancha catch a real bug in which thread it answers, read the evidence, and prove the fix.

This guide tests a small Slack support bot built with `@slack/web-api`. You write a scene with a #support channel, two questions and checks for the right reply, run the bot three times, find a bug from the evidence, and prove the fix. It takes about ten minutes and needs no Slack workspace and no API keys.

## Before you start

* Node.js 24 or later.
* The Ancha CLI: `npm install -g @ancha-labs/cli`. See [Quickstart](https://docs.ancha.io/quickstart.md).

## 1. Create the bot

```bash theme={null}
mkdir slack-bot && cd slack-bot
npm init -y
npm pkg set type=module
npm install @slack/web-api
```

Save this as `bot.mjs`. It reads #support, picks a question, and replies in its thread with an answer from a small playbook. It has a bug that Ancha will find.

```js bot.mjs theme={null}
// A small support bot: answers the newest question in #support, in its thread.
import { WebClient } from '@slack/web-api';

const slack = new WebClient(process.env.SLACK_BOT_TOKEN, { slackApiUrl: process.env.SLACK_API_URL });

const playbook = [
  { topic: /invite/i, answer: 'An admin can invite a teammate from Settings → Members → Invite.' },
  { topic: /csv|export/i, answer: 'Reports → Exports, choose the month, then Download CSV. It can take two minutes to appear.' },
];

const { channels } = await slack.conversations.list();
const support = channels.find((channel) => channel.name === 'support');
const { messages } = await slack.conversations.history({ channel: support.id }); // newest first

const question = messages.findLast((message) => message.text.endsWith('?'));
const entry = playbook.find((item) => item.topic.test(question.text));
const answer = entry ? entry.answer : 'Thanks, a teammate will follow up shortly.';

await slack.chat.postMessage({ channel: support.id, thread_ts: question.ts, text: answer });
console.log(`Answered: ${question.text}`);
```

The bot reads `SLACK_API_URL` and `SLACK_BOT_TOKEN`, so the same code runs against real Slack when you give it real values.

## 2. Set up Ancha

```bash theme={null}
ancha init --agent "node bot.mjs"
rm -r scenes/starter
```

`init` writes `ancha.yaml` with your command, a starter scene and `.ancha/` in `.gitignore`. Because `package.json` depends on `@slack/web-api`, the starter only checks that the bot reaches Slack. Remove it; you will write a scene for this bot.

## 3. Write the scene

Save this as `scenes/support-answer/scene.yaml`:

```yaml scenes/support-answer/scene.yaml theme={null}
id: support-answer
title: Answer the newest question in #support
twins: [slack]
world: static
start_time: 2026-10-26T14:00:00Z
timeout: 1m
task: Answer the newest question in Slack #support, in its thread.
seed:
  slack:
    team: { name: Acme }
    bot: $bot
    users:
      - { ref: bot, name: ancha, real_name: Ancha assistant, is_bot: true }
      - { ref: ben, name: ben, real_name: Ben Carter }
      - { ref: mike, name: mike, real_name: Mike Turner }
    channels:
      - { ref: support, name: support, members: [$bot, $ben, $mike] }
    messages:
      - ref: csv_question
        channel: $support
        user: $mike
        at: -35m
        text: "Ridgeway Robotics cannot find last month's CSV export. Where should I point them?"
      - channel: $support
        user: $ben
        at: -32m
        thread_of: $csv_question
        text: "Reports → Exports. Choose the month, then Download CSV."
      - ref: invite_question
        channel: $support
        user: $mike
        at: -2m
        text: "Ridgeway Robotics wants to add a teammate. How do they invite someone?"
checks:
  - id: answered_newest
    state:
      slack.message:
        channel: $support
        from: $bot
        in_thread_of: $invite_question
        text_contains: "Settings → Members → Invite"
  - id: posted_a_reply
    trace: { called: "POST /api/chat.postMessage" }
pass: [answered_newest, posted_a_reply]
```

The world has an old question that Ben already answered and a new one from two minutes ago. The checks say what a good outcome looks like:

* `answered_newest` looks at the final Slack state: the bot replied in the thread of the newest question, with the right instructions. `from: $bot` keeps Ben's earlier answer from counting.
* `posted_a_reply` looks at the bot's calls. It uses `POST` because the SDK calls every Slack method with POST.

Check the file:

```bash theme={null}
ancha scene validate scenes/support-answer/scene.yaml
```

```text Output theme={null}
ok  scenes/support-answer/scene.yaml (support-answer)
```

## 4. Run it three times

```bash theme={null}
ancha run -n 3
```

```text Output theme={null}
running locally (worlds in a server this command starts)
run_started runId=run_20261031-103344_2137d5d4f255e3a3 scenes=["support-answer"] trials=3 parallel=1 workspace=local:/path/to/slack-bot target=local
trial_started scene=support-answer trial=1 envId=imz4odkx
trial_finished scene=support-answer trial=1 passed=false valid=true warnings=[] exit=exited exitCode=0 realMs=92 failedChecks=["answered_newest"]
trial_started scene=support-answer trial=2 envId=yusbkdtu
trial_finished scene=support-answer trial=2 passed=false valid=true warnings=[] exit=exited exitCode=0 realMs=86 failedChecks=["answered_newest"]
trial_started scene=support-answer trial=3 envId=jv8tr2mp
trial_finished scene=support-answer trial=3 passed=false valid=true warnings=[] exit=exited exitCode=0 realMs=82 failedChecks=["answered_newest"]

✗ support-answer  0/3 (0%)
    ✗ answered_newest  failed in 3/3 trials
      trial 1: The check wanted ≥1 message in #support from @ancha in reply to @mike: "Ridgeway Robotics wants to add a teammate. How…" containing "Settings → Members → Invite" but found 0 matching; @ancha has 1 message(s) in #support, 0 in that thread.

agent not sealed: network open; environment limited to 9 variables from this shell (pass more with --pass-env NAME)
fail (minimum pass rate 100%)  run run_20261031-103344_2137d5d4f255e3a3
next: ancha explain run_20261031-103344_2137d5d4f255e3a3
```

The command exits with code `1`. Each trial ran on a fresh copy of the channel, and all three failed the same way. The bot got only Ancha's variables and a small base set from your shell, which is all it needs.

## 5. Find out why

```bash theme={null}
ancha explain latest --trial 1
```

```text Output theme={null}
support-answer  trial 1  ✗ answered_newest
  expected: ≥1 message in #support from @ancha in reply to @mike: "Ridgeway Robotics wants to add a teammate. How…" containing "Settings → Members → Invite"
  actual:   0 matching; @ancha has 1 message(s) in #support, 0 in that thread
  why:      The check wanted ≥1 message in #support from @ancha in reply to @mike: "Ridgeway Robotics wants to add a teammate. How…" containing "Settings → Members → Invite" but found 0 matching; @ancha has 1 message(s) in #support, 0 in that thread.
  trace:
    #1 agent POST /api/conversations.list -> 200
    #2 agent POST /api/conversations.history -> 200
    #3 agent POST /api/chat.postMessage -> 200

environment: 70 variables from this shell were withheld from the agent. If it needs one (a provider key, say), pass it with --pass-env NAME or list it under agent.pass_env in ancha.yaml.

next: ancha baseline save before --from run_20261031-103344_2137d5d4f255e3a3, then fix the agent, then ancha run --compare before
```

Without `--trial 1`, `explain` prints all three trials.

The bot did post one message, just not in the newest question's thread. The trial's files show where it went:

```bash theme={null}
RUN=$(cat .ancha/runs/latest)
cat .ancha/runs/$RUN/trial-support-answer-1/agent.log
grep chat.postMessage .ancha/runs/$RUN/trial-support-answer-1/trace.jsonl
```

`agent.log` says `Answered: Ridgeway Robotics cannot find last month's CSV export. Where should I point them?`, and the request in `trace.jsonl` carries that question's `thread_ts`. The bot answered the oldest question. `conversations.history` returns messages newest first, and `findLast` takes the last match, which is the oldest.

## 6. Fix it and prove the fix

Save the failing run as the reference, then fix the bot:

```bash theme={null}
ancha baseline save before
```

In `bot.mjs`, change `messages.findLast(` to `messages.find(` so the bot takes the newest question. Then run again and compare:

```bash theme={null}
ancha run -n 3 --compare before
```

```text Output theme={null}
✓ support-answer  3/3 (100%)

compared with run_20261031-103344_2137d5d4f255e3a3: no regression
  support-answer: 0% → 100% (+100 pp); 3 → 3 trials
    improved: answered_newest  0/3 → 3/3
agent not sealed: network open; environment limited to 9 variables from this shell (pass more with --pass-env NAME)
pass (minimum pass rate 100%)  run run_20261031-103345_81849c0329dfa0fd
```

Exit code `0`: every check passed and no scene and no check got worse than `before`.

## Next steps

* **Grade the wording.** Add a judge check, and keep it out of `pass` while you tune it. Without `OPENROUTER_API_KEY` it is skipped with a warning and does not change the verdict:

  ```yaml theme={null}
  - id: helpful
    judge: "The reply tells the person exactly where in the product to invite a teammate."
  ```

* **Cover more cases.** Add scenes for a question the playbook does not know, a question in a thread, or no question at all. Each scene is a file under `scenes/`.

* **Test a model-based bot.** Its answers vary, so run more trials and set `min_pass_rate` in `ancha.yaml`. List its provider key under `agent.pass_env`; Ancha withholds the rest of your shell from the bot. See [ancha.yaml](https://docs.ancha.io/reference/ancha-yaml.md).

* **React to events.** A bot that listens for messages reads Slack events from the trial's event feed, or by HTTP or Socket Mode. See [Slack](https://docs.ancha.io/tools/slack.md#events).

* **Run it in CI.** See [CI](https://docs.ancha.io/cloud/ci.md).

* **Hand it to a coding agent.** See [For coding agents](https://docs.ancha.io/coding-agents.md).


## Related topics

- [Quickstart](https://docs.ancha.io/quickstart.md)
- [Slack](https://docs.ancha.io/tools/slack.md)
- [Test file (scene.yaml)](https://docs.ancha.io/reference/test-file.md)
- [Troubleshooting](https://docs.ancha.io/troubleshooting.md)
- [ancha mcp](https://docs.ancha.io/cli/mcp.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.