Tests
The scenarios a version must pass, drafted or written, and the runs that check it before your users see it.
RoutesOpen one for every field it takes and answers.
GET/v1/deployments/{deployment_id}/scenariosThe deployment's test scenarios.
Request1 field
Path
deployment_idstringrequiredThe deployment's id (dep_...). The /v1/deployments routes also take its name.
- Type
string
Answers 2002 fields
dataobject[]The items on this page.
namestringIts name, 1 to 120 characters.
- Limits
- 1 to 120 characters.
- Example
Bag allowance
personastringWho the test user is, in a few sentences, up to 2,000 characters.
- Limits
- 1 to 2000 characters.
- Example
A first-time flyer, anxious, on a phone.
goalstringWhat the test user wants, in their words, up to 2,000 characters.
- Limits
- 1 to 2000 characters.
- Example
How many kilos can I take on booking TN-48213?
success_criteriastring[]What a pass looks like: 1 to 12 lines, each up to 500 characters. Every one must be met.
- Limits
- 1 to 12 items.
- Example
["Says 7 kg of cabin baggage."]
must_neverstring[]What must never happen: up to 12 lines, each up to 500 characters. Any one fails the test.
- Default
[]- Limits
- At most 12 items.
- Example
["Promises a refund."]
tool_resultsmap of anymay be absentWhat each tool answers in this test, as any JSON, at most 20: a declared tool by its name, and a tool of an MCP server as <server>.<tool> (the server's slug), since a test never runs one for real unless the server marks it read-only. A tool with none answers an error.
- Example
{"get_booking":{"cabin_kg":7}}
approvesbooleanmay be absenttrue: the test user says yes to each approval for a tool of one of the deployment's own MCP servers, which in a test plays its tool_results entry or does not run. Absent or false: it says no to every approval. Anything else that waits for a yes is always declined.
- Example
true
idstringThe scenario's id.
- Type
string
object"scenario"Always scenario.
- Type
"scenario"
deployment_idstringThe deployment it tests.
- Type
string
created_atstringWhen it was made, as an ISO 8601 date and time in UTC.
- Type
string
updated_atstringWhen it last changed, as an ISO 8601 date and time in UTC.
- Type
string
next_cursorstring | nullSend it as cursor for the next page; null on the last page.
- Type
string | null
No sample in your language for this route, so here it is over HTTP.
curl "$VESTRA_BASE_URL/v1/deployments/{deployment_id}/scenarios" \ -H "Authorization: Bearer $VESTRA_API_KEY"POST/v1/deployments/{deployment_id}/scenariosAdd one: a pretend user, their goal, what success looks like, what must never happen.
POST/v1/deployments/{deployment_id}/scenarios/generateHave Bash draft scenarios from the deployment's instructions.
PATCH/v1/deployments/{deployment_id}/scenarios/{scenario_id}Change one.
DELETE/v1/deployments/{deployment_id}/scenarios/{scenario_id}Remove one.
POST/v1/deployments/{deployment_id}/test-runsRun every scenario against a version: version, or the latest when left out.
The deployment is named by its id or its name. user_secrets sets the test user's own secrets for your MCP servers, replacing any an earlier run set.