# Tests API

> The scenarios a version must pass, drafted or written, and the runs that check it before your users see it.

Send `Authorization: Bearer vk_test_...` on every call.

## `GET /v1/deployments/{deployment_id}/scenarios`

Its own page: [GET /v1/deployments/{deployment_id}/scenarios](https://vestra.ai/developers/reference/api/tests/get-v1-deployments-deployment-id-scenarios.md)

The deployment's test scenarios.

Your API key, scope `setup` or `all`. On the card: [`test`](https://vestra.ai/developers/test.md). Over HTTP it is made from the contract: fill in each {…} and every value.

### Request

Path:

- `deployment_id` (`string`, required): The deployment's id (`dep_...`). The /v1/deployments routes also take its name.

### Answers 200

- `data` (`object[]`): The items on this page.
  - `name` (`string`): Its name, 1 to 120 characters. Limits: 1 to 120 characters. Example: `Bag allowance`.
  - `persona` (`string`): Who the test user is, in a few sentences, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `A first-time flyer, anxious, on a phone.`.
  - `goal` (`string`): What the test user wants, in their words, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `How many kilos can I take on booking TN-48213?`.
  - `success_criteria` (`string[]`): What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met. Limits: 1 to 12 items. Example: `["Says 7 kg of cabin baggage."]`.
  - `must_never` (`string[]`): What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test. Default: `[]`. Limits: at most 12 items. Example: `["Promises a refund."]`.
  - `tool_results` (`map of any`, may be absent): What each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error. Example: `{"get_booking":{"cabin_kg":7}}`.
  - `approves` (`boolean`, may be absent): true: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined. Example: `true`.
  - `id` (`string`): The scenario's id.
  - `object` (`"scenario"`): Always `scenario`.
  - `deployment_id` (`string`): The deployment it tests.
  - `created_at` (`string`): When it was made, as an ISO 8601 date and time in UTC.
  - `updated_at` (`string`): When it last changed, as an ISO 8601 date and time in UTC.
- `next_cursor` (`string | null`): Send it as `cursor` for the next page; null on the last page.

### Code

**HTTP**

```bash
curl "$VESTRA_BASE_URL/v1/deployments/{deployment_id}/scenarios" \
  -H "Authorization: Bearer $VESTRA_API_KEY"
```

**TypeScript, Python, CLI, MCP**: No sample in your language for this route, so here it is over HTTP.

## `POST /v1/deployments/{deployment_id}/scenarios`

Its own page: [POST /v1/deployments/{deployment_id}/scenarios](https://vestra.ai/developers/reference/api/tests/post-v1-deployments-deployment-id-scenarios.md)

Add one: a pretend user, their goal, what success looks like, what must never happen.

Your API key, scope `setup` or `all`. On the card: [`test`](https://vestra.ai/developers/test.md). Over HTTP it is made from the contract: fill in each {…} and every value.

### Request

Path:

- `deployment_id` (`string`, required): The deployment's id (`dep_...`). The /v1/deployments routes also take its name.

Headers:

- `Idempotency-Key` (`string`): Send the same key to retry this call safely for 24 hours.

Body:

- `name` (`string`, required): Its name, 1 to 120 characters. Limits: 1 to 120 characters. Example: `Bag allowance`.
- `persona` (`string`, required): Who the test user is, in a few sentences, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `A first-time flyer, anxious, on a phone.`.
- `goal` (`string`, required): What the test user wants, in their words, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `How many kilos can I take on booking TN-48213?`.
- `success_criteria` (`string[]`, required): What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met. Limits: 1 to 12 items. Example: `["Says 7 kg of cabin baggage."]`.
- `must_never` (`string[]`): What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test. Default: `[]`. Limits: at most 12 items. Example: `["Promises a refund."]`.
- `tool_results` (`map of any`): What each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error. Example: `{"get_booking":{"cabin_kg":7}}`.
- `approves` (`boolean`): true: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined. Example: `true`.

### Answers 201

- `name` (`string`): Its name, 1 to 120 characters. Limits: 1 to 120 characters. Example: `Bag allowance`.
- `persona` (`string`): Who the test user is, in a few sentences, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `A first-time flyer, anxious, on a phone.`.
- `goal` (`string`): What the test user wants, in their words, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `How many kilos can I take on booking TN-48213?`.
- `success_criteria` (`string[]`): What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met. Limits: 1 to 12 items. Example: `["Says 7 kg of cabin baggage."]`.
- `must_never` (`string[]`): What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test. Default: `[]`. Limits: at most 12 items. Example: `["Promises a refund."]`.
- `tool_results` (`map of any`, may be absent): What each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error. Example: `{"get_booking":{"cabin_kg":7}}`.
- `approves` (`boolean`, may be absent): true: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined. Example: `true`.
- `id` (`string`): The scenario's id.
- `object` (`"scenario"`): Always `scenario`.
- `deployment_id` (`string`): The deployment it tests.
- `created_at` (`string`): When it was made, as an ISO 8601 date and time in UTC.
- `updated_at` (`string`): When it last changed, as an ISO 8601 date and time in UTC.

### Code

**HTTP**

```bash
curl -X POST "$VESTRA_BASE_URL/v1/deployments/{deployment_id}/scenarios" \
  -H "Authorization: Bearer $VESTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Bag allowance",
    "persona": "A first-time flyer, anxious, on a phone.",
    "goal": "How many kilos can I take on booking TN-48213?",
    "success_criteria": [
      "Says 7 kg of cabin baggage."
    ]
  }'
```

**TypeScript, Python, CLI, MCP**: No sample in your language for this route, so here it is over HTTP.

## `POST /v1/deployments/{deployment_id}/scenarios/generate`

Its own page: [POST /v1/deployments/{deployment_id}/scenarios/generate](https://vestra.ai/developers/reference/api/tests/post-v1-deployments-deployment-id-scenarios-generate.md)

Have Bash draft scenarios from the deployment's instructions.

Your API key, scope `setup` or `all`. On the card: [`test`](https://vestra.ai/developers/test.md). Every call here is a checked sample.

### Request

Path:

- `deployment_id` (`string`, required): The deployment's id (`dep_...`). The /v1/deployments routes also take its name.

Headers:

- `Idempotency-Key` (`string`): Send the same key to retry this call safely for 24 hours.

Body, if you send one:

- `count` (`integer`): How many tests to draft, 1 to 10. Default 5. Default: `5`. Limits: 1 to 10.
- `version` (`integer`): The version whose instructions they are drafted from. Absent: the newest. Limits: at least 1.

### Answers 200

- `object` (`"list"`): Always `list`.
- `data` (`object[]`): The drafted tests. They are not saved: save the ones you want, or write them into vestra.yaml.
  - `name` (`string`): Its name, 1 to 120 characters. Limits: 1 to 120 characters. Example: `Bag allowance`.
  - `persona` (`string`): Who the test user is, in a few sentences, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `A first-time flyer, anxious, on a phone.`.
  - `goal` (`string`): What the test user wants, in their words, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `How many kilos can I take on booking TN-48213?`.
  - `success_criteria` (`string[]`): What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met. Limits: 1 to 12 items. Example: `["Says 7 kg of cabin baggage."]`.
  - `must_never` (`string[]`): What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test. Default: `[]`. Limits: at most 12 items. Example: `["Promises a refund."]`.
  - `tool_results` (`map of any`, may be absent): What each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error. Example: `{"get_booking":{"cabin_kg":7}}`.
  - `approves` (`boolean`, may be absent): true: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined. Example: `true`.

### Code

**TypeScript**

```ts
const drafted = await vestra.test.generate('vestra.yaml');
```

**Python**

```python
drafted = vestra.test.generate("vestra.yaml")
```

**CLI**

```bash
vestra test --generate
```

**HTTP**

```bash
# Draft more tests. The tests in your file are the deployment's scenarios, and Bash can draft more from your
# instructions.
curl "$VESTRA_BASE_URL/v1/deployments/dep_4kq/scenarios/generate" \
  -H "Authorization: Bearer $VESTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "count": 3 }'

# Run the tests. This plays every scenario against the latest version, by the deployment's name. Then read
# each verdict. If a restart cuts a run off, it resumes on its own: resumes says how often, and error_code says
# why a run ended in error.
curl -X POST "$VESTRA_BASE_URL/v1/deployments/trip-rescue/test-runs" \
  -H "Authorization: Bearer $VESTRA_API_KEY"

curl "$VESTRA_BASE_URL/v1/test-runs/trun_2k1" \
  -H "Authorization: Bearer $VESTRA_API_KEY"
```

**MCP**

```json
{
  "method": "tools/call",
  "params": {
    "name": "test",
    "arguments": { "deployment": "trip-rescue" }
  }
}
```

## `PATCH /v1/deployments/{deployment_id}/scenarios/{scenario_id}`

Its own page: [PATCH /v1/deployments/{deployment_id}/scenarios/{scenario_id}](https://vestra.ai/developers/reference/api/tests/patch-v1-deployments-deployment-id-scenarios-scenario-id.md)

Change one.

Your API key, scope `setup` or `all`. On the card: [`test`](https://vestra.ai/developers/test.md). Over HTTP it is made from the contract: fill in each {…} and every value.

### Request

Path:

- `deployment_id` (`string`, required): The deployment's id (`dep_...`). The /v1/deployments routes also take its name.
- `scenario_id` (`string`, required): The test scenario's id (`scn_...`).

Body:

- `name` (`string`): Its name, 1 to 120 characters. Limits: 1 to 120 characters. Example: `Bag allowance`.
- `persona` (`string`): Who the test user is, in a few sentences, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `A first-time flyer, anxious, on a phone.`.
- `goal` (`string`): What the test user wants, in their words, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `How many kilos can I take on booking TN-48213?`.
- `success_criteria` (`string[]`): What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met. Limits: 1 to 12 items. Example: `["Says 7 kg of cabin baggage."]`.
- `must_never` (`string[]`): What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test. Limits: at most 12 items. Example: `["Promises a refund."]`.
- `tool_results` (`map of any`): What each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error. Example: `{"get_booking":{"cabin_kg":7}}`.
- `approves` (`boolean`): true: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined. Example: `true`.

### Answers 200

- `name` (`string`): Its name, 1 to 120 characters. Limits: 1 to 120 characters. Example: `Bag allowance`.
- `persona` (`string`): Who the test user is, in a few sentences, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `A first-time flyer, anxious, on a phone.`.
- `goal` (`string`): What the test user wants, in their words, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `How many kilos can I take on booking TN-48213?`.
- `success_criteria` (`string[]`): What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met. Limits: 1 to 12 items. Example: `["Says 7 kg of cabin baggage."]`.
- `must_never` (`string[]`): What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test. Default: `[]`. Limits: at most 12 items. Example: `["Promises a refund."]`.
- `tool_results` (`map of any`, may be absent): What each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error. Example: `{"get_booking":{"cabin_kg":7}}`.
- `approves` (`boolean`, may be absent): true: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined. Example: `true`.
- `id` (`string`): The scenario's id.
- `object` (`"scenario"`): Always `scenario`.
- `deployment_id` (`string`): The deployment it tests.
- `created_at` (`string`): When it was made, as an ISO 8601 date and time in UTC.
- `updated_at` (`string`): When it last changed, as an ISO 8601 date and time in UTC.

### Code

**HTTP**

```bash
curl -X PATCH "$VESTRA_BASE_URL/v1/deployments/{deployment_id}/scenarios/{scenario_id}" \
  -H "Authorization: Bearer $VESTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}'
```

**TypeScript, Python, CLI, MCP**: No sample in your language for this route, so here it is over HTTP.

## `DELETE /v1/deployments/{deployment_id}/scenarios/{scenario_id}`

Its own page: [DELETE /v1/deployments/{deployment_id}/scenarios/{scenario_id}](https://vestra.ai/developers/reference/api/tests/delete-v1-deployments-deployment-id-scenarios-scenario-id.md)

Remove one.

Your API key, scope `setup` or `all`. On the card: [`test`](https://vestra.ai/developers/test.md). Over HTTP it is made from the contract: fill in each {…} and every value.

### Request

Path:

- `deployment_id` (`string`, required): The deployment's id (`dep_...`). The /v1/deployments routes also take its name.
- `scenario_id` (`string`, required): The test scenario's id (`scn_...`).

### Answers 200

- `id` (`string`): The deleted scenario's id.
- `object` (`"scenario"`): Always `scenario`.
- `deleted` (`true`): Always true: no test run plays it again.

### Code

**HTTP**

```bash
curl -X DELETE "$VESTRA_BASE_URL/v1/deployments/{deployment_id}/scenarios/{scenario_id}" \
  -H "Authorization: Bearer $VESTRA_API_KEY"
```

**TypeScript, Python, CLI, MCP**: No sample in your language for this route, so here it is over HTTP.

## `POST /v1/deployments/{deployment_id}/test-runs`

Its own page: [POST /v1/deployments/{deployment_id}/test-runs](https://vestra.ai/developers/reference/api/tests/post-v1-deployments-deployment-id-test-runs.md)

Run every scenario against a version: version, or the latest when left out. The deployment is named by its id or its name. user_secrets sets the test user's own secrets for your MCP servers, replacing any an earlier run set.

Your API key, scope `setup` or `all`. On the card: [`test`](https://vestra.ai/developers/test.md). Every call here is a checked sample.

### Request

Path:

- `deployment_id` (`string`, required): The deployment's id (`dep_...`). The /v1/deployments routes also take its name.

Headers:

- `Idempotency-Key` (`string`): Send the same key to retry this call safely for 24 hours.

Body, if you send one:

- `version` (`integer`): The version to test. Absent: the newest. Limits: at least 1.
- `user_secrets` (`map of string`): The test user's own secrets, by name, at most 20, for headers such as `${user_secret:NAME}`. They replace the whole set: `{}` clears it, and a run that sends none keeps the last. Never returned. Example: `{"AGENCY_ID":"..."}`.

### Answers 202

- `id` (`string`): The test run's id. Example: `trun_8f2k`.
- `object` (`"test_run"`): Always `test_run`.
- `deployment_id` (`string`): The deployment it tests.
- `version` (`integer`): The version it tests.
- `status` (`"queued" | "running" | "passed" | "failed" | "error"`): `queued` or `running` while it plays. `passed`: every scenario passed. `failed`: one or more did not. `error`: it could not finish, as `error_code` says.
- `error_code` (`string | null`): Why it ended in `error`: `interrupted` (a restart stopped it too often), `test_wallet_empty`, `judge_unreadable` (the grader's answer could not be read) or `internal`. Null otherwise.
- `scenario_count` (`integer`): How many scenarios it plays.
- `passed_count` (`integer`): How many have passed so far.
- `resumes` (`integer`): How many times a restart interrupted it and it played on from where it was, at most 2.
- `promoted` (`object | null`): What its pass promoted, for an apply that promotes after its tests; null when it promoted nothing (none asked, refused, or it did not pass).
  - `mode` (`"live" | "test"`): The mode the version went live in.
  - `version` (`integer`): The version that went live.
- `became_default` (`boolean`): True when that promotion made the deployment the company's default: an ask with no deployment now reaches it.
- `cost_usd` (`number`): What the run cost, in billed US dollars.
- `created_at` (`string`): When it was asked for, as an ISO 8601 date and time in UTC.
- `started_at` (`string | null`): When it began to play; null while queued.
- `finished_at` (`string | null`): When it ended; null while it plays.
- `results` (`object[]`, may be absent): Each scenario's result, on GET /v1/test-runs/{test_run_id}; absent in lists.
  - `object` (`"test_result"`): Always `test_result`.
  - `scenario_id` (`string | null`): The scenario it played; null when the scenario was deleted since.
  - `scenario` (`object`): The scenario as it was played.
    - `name` (`string`): Its name, 1 to 120 characters. Limits: 1 to 120 characters. Example: `Bag allowance`.
    - `persona` (`string`): Who the test user is, in a few sentences, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `A first-time flyer, anxious, on a phone.`.
    - `goal` (`string`): What the test user wants, in their words, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `How many kilos can I take on booking TN-48213?`.
    - `success_criteria` (`string[]`): What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met. Limits: 1 to 12 items. Example: `["Says 7 kg of cabin baggage."]`.
    - `must_never` (`string[]`): What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test. Default: `[]`. Limits: at most 12 items. Example: `["Promises a refund."]`.
    - `tool_results` (`map of any`, may be absent): What each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error. Example: `{"get_booking":{"cabin_kg":7}}`.
    - `approves` (`boolean`, may be absent): true: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined. Example: `true`.
  - `verdict` (`"passed" | "failed" | "error"`): `passed`: every line of success met and nothing it must never do. `failed`: not so. `error`: it could not be played to the end.
  - `reason` (`string`): Why, in a sentence or two.
  - `criteria` (`object[]`): Each line of success, graded.
    - `criterion` (`string`): One line of what a pass looks like.
    - `met` (`boolean`): Whether the conversation met it.
    - `evidence` (`string`): Why the grader says so, from the conversation.
  - `must_never` (`object[]`): Each thing it must never do, graded.
    - `rule` (`string`): One line of what must never happen.
    - `violated` (`boolean`): Whether it happened.
    - `evidence` (`string`): Why the grader says so, from the conversation.
  - `approvals_asked` (`object[]`): What Bash asked the test user's yes for, and what it said.
    - `kind` (`string`): The kind of ask Bash raised, such as `hands_approval`, `credit_approval` or `tool_permission`.
    - `category` (`"payment" | "account_action" | "connect_app" | "action" | "custom"`): What it would have done: a `payment`, an `account_action`, `connect_app`, another `action`, or `custom`.
    - `summary` (`string`): What it asked to do, in its words.
    - `decision` (`"denied" | "approved"`): What the test user said. It denies anything that would act for it, so a test never moves money or changes an account; a test with approves: true approves the tools of the deployment's own MCP servers, which a test plays from its tool_results or does not run.
  - `transcript` (`object[]`): The conversation, line by line.
    - `role` (`"customer" | "bash" | "note"`): Who said it: `customer` (the test user), `bash`, or a `note` on what happened (work started, a tool called).
    - `text` (`string`): What was said, or what happened.
  - `conversation_id` (`string | null`): The test conversation; null when it never started.
  - `cost_usd` (`number`): What playing it cost, in billed US dollars.

### Code

**TypeScript**

```ts
const summary = await vestra.test('vestra.yaml');
```

**Python**

```python
summary = vestra.test("vestra.yaml")
```

**CLI**

```bash
vestra test
```

**HTTP**

```bash
# Draft more tests. The tests in your file are the deployment's scenarios, and Bash can draft more from your
# instructions.
curl "$VESTRA_BASE_URL/v1/deployments/dep_4kq/scenarios/generate" \
  -H "Authorization: Bearer $VESTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "count": 3 }'

# Run the tests. This plays every scenario against the latest version, by the deployment's name. Then read
# each verdict. If a restart cuts a run off, it resumes on its own: resumes says how often, and error_code says
# why a run ended in error.
curl -X POST "$VESTRA_BASE_URL/v1/deployments/trip-rescue/test-runs" \
  -H "Authorization: Bearer $VESTRA_API_KEY"

curl "$VESTRA_BASE_URL/v1/test-runs/trun_2k1" \
  -H "Authorization: Bearer $VESTRA_API_KEY"
```

**MCP**

```json
{
  "method": "tools/call",
  "params": {
    "name": "test",
    "arguments": { "deployment": "trip-rescue" }
  }
}
```

## `GET /v1/deployments/{deployment_id}/test-runs`

Its own page: [GET /v1/deployments/{deployment_id}/test-runs](https://vestra.ai/developers/reference/api/tests/get-v1-deployments-deployment-id-test-runs.md)

Past test runs.

Your API key, scope `setup` or `all`. On the card: [`test`](https://vestra.ai/developers/test.md). Over HTTP it is made from the contract: fill in each {…} and every value.

### Request

Path:

- `deployment_id` (`string`, required): The deployment's id (`dep_...`). The /v1/deployments routes also take its name.

Query:

- `limit` (`integer`): How many items per page, 1 to 100. Default 20.
- `cursor` (`string`): Where to go on from: the `next_cursor` of the page before, unchanged. Absent: the first page.

### Answers 200

- `data` (`object[]`): The items on this page.
  - `id` (`string`): The test run's id. Example: `trun_8f2k`.
  - `object` (`"test_run"`): Always `test_run`.
  - `deployment_id` (`string`): The deployment it tests.
  - `version` (`integer`): The version it tests.
  - `status` (`"queued" | "running" | "passed" | "failed" | "error"`): `queued` or `running` while it plays. `passed`: every scenario passed. `failed`: one or more did not. `error`: it could not finish, as `error_code` says.
  - `error_code` (`string | null`): Why it ended in `error`: `interrupted` (a restart stopped it too often), `test_wallet_empty`, `judge_unreadable` (the grader's answer could not be read) or `internal`. Null otherwise.
  - `scenario_count` (`integer`): How many scenarios it plays.
  - `passed_count` (`integer`): How many have passed so far.
  - `resumes` (`integer`): How many times a restart interrupted it and it played on from where it was, at most 2.
  - `promoted` (`object | null`): What its pass promoted, for an apply that promotes after its tests; null when it promoted nothing (none asked, refused, or it did not pass).
    - `mode` (`"live" | "test"`): The mode the version went live in.
    - `version` (`integer`): The version that went live.
  - `became_default` (`boolean`): True when that promotion made the deployment the company's default: an ask with no deployment now reaches it.
  - `cost_usd` (`number`): What the run cost, in billed US dollars.
  - `created_at` (`string`): When it was asked for, as an ISO 8601 date and time in UTC.
  - `started_at` (`string | null`): When it began to play; null while queued.
  - `finished_at` (`string | null`): When it ended; null while it plays.
  - `results` (`object[]`, may be absent): Each scenario's result, on GET /v1/test-runs/{test_run_id}; absent in lists.
    - `object` (`"test_result"`): Always `test_result`.
    - `scenario_id` (`string | null`): The scenario it played; null when the scenario was deleted since.
    - `scenario` (`object`): The scenario as it was played.
    - `verdict` (`"passed" | "failed" | "error"`): `passed`: every line of success met and nothing it must never do. `failed`: not so. `error`: it could not be played to the end.
    - `reason` (`string`): Why, in a sentence or two.
    - `criteria` (`object[]`): Each line of success, graded.
    - `must_never` (`object[]`): Each thing it must never do, graded.
    - `approvals_asked` (`object[]`): What Bash asked the test user's yes for, and what it said.
    - `transcript` (`object[]`): The conversation, line by line.
    - `conversation_id` (`string | null`): The test conversation; null when it never started.
    - `cost_usd` (`number`): What playing it cost, in billed US dollars.
- `next_cursor` (`string | null`): Send it as `cursor` for the next page; null on the last page.

### Code

**HTTP**

```bash
curl "$VESTRA_BASE_URL/v1/deployments/{deployment_id}/test-runs" \
  -H "Authorization: Bearer $VESTRA_API_KEY"
```

**TypeScript, Python, CLI, MCP**: No sample in your language for this route, so here it is over HTTP.

## `GET /v1/test-runs/{test_run_id}`

Its own page: [GET /v1/test-runs/{test_run_id}](https://vestra.ai/developers/reference/api/tests/get-v1-test-runs-test-run-id.md)

Pass or fail per scenario, the conversation and the judge's reason.

Your API key, scope `setup` or `all`. On the card: [`test`](https://vestra.ai/developers/test.md). Every call here is a checked sample.

### Request

Path:

- `test_run_id` (`string`, required): The test run's id (`trun_...`).

### Answers 200

- `id` (`string`): The test run's id. Example: `trun_8f2k`.
- `object` (`"test_run"`): Always `test_run`.
- `deployment_id` (`string`): The deployment it tests.
- `version` (`integer`): The version it tests.
- `status` (`"queued" | "running" | "passed" | "failed" | "error"`): `queued` or `running` while it plays. `passed`: every scenario passed. `failed`: one or more did not. `error`: it could not finish, as `error_code` says.
- `error_code` (`string | null`): Why it ended in `error`: `interrupted` (a restart stopped it too often), `test_wallet_empty`, `judge_unreadable` (the grader's answer could not be read) or `internal`. Null otherwise.
- `scenario_count` (`integer`): How many scenarios it plays.
- `passed_count` (`integer`): How many have passed so far.
- `resumes` (`integer`): How many times a restart interrupted it and it played on from where it was, at most 2.
- `promoted` (`object | null`): What its pass promoted, for an apply that promotes after its tests; null when it promoted nothing (none asked, refused, or it did not pass).
  - `mode` (`"live" | "test"`): The mode the version went live in.
  - `version` (`integer`): The version that went live.
- `became_default` (`boolean`): True when that promotion made the deployment the company's default: an ask with no deployment now reaches it.
- `cost_usd` (`number`): What the run cost, in billed US dollars.
- `created_at` (`string`): When it was asked for, as an ISO 8601 date and time in UTC.
- `started_at` (`string | null`): When it began to play; null while queued.
- `finished_at` (`string | null`): When it ended; null while it plays.
- `results` (`object[]`, may be absent): Each scenario's result, on GET /v1/test-runs/{test_run_id}; absent in lists.
  - `object` (`"test_result"`): Always `test_result`.
  - `scenario_id` (`string | null`): The scenario it played; null when the scenario was deleted since.
  - `scenario` (`object`): The scenario as it was played.
    - `name` (`string`): Its name, 1 to 120 characters. Limits: 1 to 120 characters. Example: `Bag allowance`.
    - `persona` (`string`): Who the test user is, in a few sentences, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `A first-time flyer, anxious, on a phone.`.
    - `goal` (`string`): What the test user wants, in their words, up to 2,000 characters. Limits: 1 to 2000 characters. Example: `How many kilos can I take on booking TN-48213?`.
    - `success_criteria` (`string[]`): What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met. Limits: 1 to 12 items. Example: `["Says 7 kg of cabin baggage."]`.
    - `must_never` (`string[]`): What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test. Default: `[]`. Limits: at most 12 items. Example: `["Promises a refund."]`.
    - `tool_results` (`map of any`, may be absent): What each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error. Example: `{"get_booking":{"cabin_kg":7}}`.
    - `approves` (`boolean`, may be absent): true: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined. Example: `true`.
  - `verdict` (`"passed" | "failed" | "error"`): `passed`: every line of success met and nothing it must never do. `failed`: not so. `error`: it could not be played to the end.
  - `reason` (`string`): Why, in a sentence or two.
  - `criteria` (`object[]`): Each line of success, graded.
    - `criterion` (`string`): One line of what a pass looks like.
    - `met` (`boolean`): Whether the conversation met it.
    - `evidence` (`string`): Why the grader says so, from the conversation.
  - `must_never` (`object[]`): Each thing it must never do, graded.
    - `rule` (`string`): One line of what must never happen.
    - `violated` (`boolean`): Whether it happened.
    - `evidence` (`string`): Why the grader says so, from the conversation.
  - `approvals_asked` (`object[]`): What Bash asked the test user's yes for, and what it said.
    - `kind` (`string`): The kind of ask Bash raised, such as `hands_approval`, `credit_approval` or `tool_permission`.
    - `category` (`"payment" | "account_action" | "connect_app" | "action" | "custom"`): What it would have done: a `payment`, an `account_action`, `connect_app`, another `action`, or `custom`.
    - `summary` (`string`): What it asked to do, in its words.
    - `decision` (`"denied" | "approved"`): What the test user said. It denies anything that would act for it, so a test never moves money or changes an account; a test with approves: true approves the tools of the deployment's own MCP servers, which a test plays from its tool_results or does not run.
  - `transcript` (`object[]`): The conversation, line by line.
    - `role` (`"customer" | "bash" | "note"`): Who said it: `customer` (the test user), `bash`, or a `note` on what happened (work started, a tool called).
    - `text` (`string`): What was said, or what happened.
  - `conversation_id` (`string | null`): The test conversation; null when it never started.
  - `cost_usd` (`number`): What playing it cost, in billed US dollars.

### Code

**TypeScript**

```ts
const run = await vestra.test.wait('trun_5kd2');
```

**Python**

```python
run = vestra.test.wait("trun_5kd2")
```

**CLI**

```bash
vestra events trun_5kd2
```

**HTTP**

```bash
# Draft more tests. The tests in your file are the deployment's scenarios, and Bash can draft more from your
# instructions.
curl "$VESTRA_BASE_URL/v1/deployments/dep_4kq/scenarios/generate" \
  -H "Authorization: Bearer $VESTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "count": 3 }'

# Run the tests. This plays every scenario against the latest version, by the deployment's name. Then read
# each verdict. If a restart cuts a run off, it resumes on its own: resumes says how often, and error_code says
# why a run ended in error.
curl -X POST "$VESTRA_BASE_URL/v1/deployments/trip-rescue/test-runs" \
  -H "Authorization: Bearer $VESTRA_API_KEY"

curl "$VESTRA_BASE_URL/v1/test-runs/trun_2k1" \
  -H "Authorization: Bearer $VESTRA_API_KEY"
```

**MCP**

```json
{
  "method": "tools/call",
  "params": {
    "name": "test",
    "arguments": { "deployment": "trip-rescue" }
  }
}
```