How-to guideAgentic QA scenarios

Describe behavior as bounded data

Scenarios are interpreted by Code Voucher's trusted browser runner. They are YAML data, not repository-provided Playwright, JavaScript, shell commands, or model instructions.

Test success and failure

This pair verifies the visible newsletter workflow and proves the local substitute rejects non-synthetic input. Required assertion failures fail QA; optional scenarios produce advisories.

.code-voucher.yml

qa:
  startup:
    type: docker_compose
    file: docker-compose.qa.yml
    service: web
    healthcheck_url: http://web:3000/health
    browser_url: http://web:3000
    timeout_seconds: 180
    build: true
  exploration:
    entrypoints: [/, /login, /dashboard]
    changed_routes: auto
    max_minutes: 8
    max_pages: 12
  auth:
    mode: none
  secrets:
    profile: none
    allow_on_forks: false
  scenarios:
    - id: newsletter-subscription
      title: Newsletter subscription
      route: /en
      required: true
      steps:
        - fill:
            locator: { label: Email address }
            value: [email protected]
        - click:
            locator: { role: button, name: Subscribe }
        - expect:
            text:
              locator: { test_id: newsletter-status }
              value: You're on the list.
              match: exact
        - screenshot: { name: newsletter-success }
    - id: newsletter-rejects-real-address
      title: Newsletter rejects non-synthetic input
      route: /en
      required: true
      steps:
        - fill:
            locator: { label: Email address }
            value: [email protected]
        - click:
            locator: { role: button, name: Subscribe }
        - expect:
            visible: { test_id: newsletter-error }

Use stable, accessible locators

Prefer test_id for application states and role plus accessible name for commands
Use label or placeholder for form fields
Assert visible user outcomes, not only HTTP success
Capture screenshots after the outcome is visible
Keep dynamic routes concrete in entrypoints or scenario routes
Do not use CSS, XPath, sleeps, scripts, request interception, or file operations

Implemented limits

scenarios20

steps per scenario30

steps per run200

screenshots per scenario2

screenshots per run30

characters per literal value512

Each scenario receives a fresh browser context. Cross-origin requests, popups, downloads, dialogs, service workers, additional pages, and arbitrary frame targeting are blocked by the trusted runner.

Instructions for a coding agent

Setup task

Prepare this repository for Code Voucher Agentic QA.
1. Read the app's existing development and test setup.
2. Add .code-voucher.yml and docker-compose.qa.yml using only the documented keys.
3. Make startup deterministic from an empty database.
4. Seed synthetic .test identities only.
5. Replace every external side effect with an attempt-local fail-closed service.
6. Add one required happy-path and one required failure-path scenario for each changed critical workflow.
7. Do not add credentials, host ports, host mounts, env_file, privileged mode, or the Docker socket.
8. Run repository-owned Compose only with explicit human approval, in a disposable environment that holds no credentials. Never run an untrusted setup directly on a developer workstation.
9. Report every unsupported dependency explicitly.

Review an agent-generated setup like any other untrusted code change. A future setup skill and validator can automate checks, but they do not replace repository-owner review.