Skip to documentation
Browse documentation
How-to guideAgentic QA dependencies

Replace every external side effect

QA must not charge a card, send an email, create a CRM contact, call an AI provider, or mutate a production account. Route each integration to an attempt-local substitute selected by non-secret QA configuration.

The substitution pattern

  1. 1

    List outbound dependencies

    Include payments, email, SMS, OAuth, storage, webhooks, analytics, maps, search, queues, and AI providers.
  2. 2

    Add a local service

    Run a small deterministic substitute in docker-compose.qa.yml. It receives no real provider credential.
  3. 3

    Select it through QA configuration

    Point the application at the Compose service hostname, such as http://newsletter-stub:8080.
  4. 4

    Fail closed

    Unknown endpoints and unsupported cases return an explicit error. Never return success for every request.
  5. 5

    Exercise failure paths

    High-risk integrations need both positive and negative scenarios, including retries and replay where applicable.

Minimal newsletter substitute

The application sees the same HTTP boundary it uses in production, but the substitute recognizes only the behavior required by QA. Server-side code may call it over the Compose network. Browser code must call a same-origin application route such as /api/newsletter; the trusted browser blocks direct cross-origin requests to substitute services.

qa/stubs/newsletter/server.py
from http.server import BaseHTTPRequestHandler, HTTPServer
import json

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path != "/health":
            return self.send_error(404, "unsupported operation")
        self.send_response(204)
        self.end_headers()

    def do_POST(self):
        if self.path != "/subscriptions":
            return self.send_error(404, "unsupported operation")
        length = int(self.headers.get("Content-Length", "0"))
        payload = json.loads(self.rfile.read(length))
        if not payload.get("email", "").endswith(".test"):
            return self.send_error(422, "synthetic .test email required")
        body = b'{"status":"subscribed"}'
        self.send_response(201)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

HTTPServer(("0.0.0.0", 8080), Handler).serve_forever()
qa/stubs/newsletter/Dockerfile
FROM python:3.12-alpine
WORKDIR /app
COPY server.py .
USER 65532:65532
CMD ["python", "server.py"]

Minimum failure coverage

IntegrationAt minimum, model
Paymentssuccess, decline, idempotency, duplicate and invalid-signature webhook
Identity / OAuthsuccess, denial, expiry, replay and invalid state
Email / SMSaccepted, provider rejection and retry
AI providerssuccess, timeout, rate limit, malformed output and budget exhaustion
Object storageupload, missing object, invalid type and size rejection

Current boundary: local Compose substitutes work today. Declarative dependency manifests, trusted operation introspection, and provider fidelity labels are Phase 2 and must not be added to .code-voucher.yml yet.

Review checklist

  • No production or provider sandbox endpoint is reachable
  • No real API key, webhook secret, card, phone number, or customer record is present
  • Unsupported behavior fails explicitly
  • State is reset for every attempt
  • Visible UI assertions accompany substitute responses
  • A successful substitute is not described as proof of provider conformance