Skip to main content

Producer Testing Guide

This guide covers how to use CVT for producer-side contract testing. Producer testing ensures your API implementation matches your OpenAPI specification before deployment.

Example Schema

The examples in this guide use the Petstore schema from sdks/shared/openapi.json. Copy it to your project as openapi.json or adjust the paths to reference it directly.

All registerSchema methods accept both file paths and URLs:

// From file
await validator.registerSchema("petstore", "./openapi.json");
// From URL
await validator.registerSchema(
  "petstore",
  "https://petstore3.swagger.io/api/v3/openapi.json",
);

Overview​

Producer testing answers the question: "Does my API match my spec?"

Producer Owns the Schema

As the producer, you own the OpenAPI spec — it is the source of truth for your API's contract. Your CI/CD pipeline should register the schema with the shared CVT server so that consumers can validate against it and register their dependencies. This enables deployment safety via can-i-deploy. See Schema Registration in CI/CD below.

ApproachWho Uses ItWhat It Tests
Consumer TestingAPI consumers"Can I call this API correctly?"
Producer TestingAPI producers"Does my API match my spec?"
┌─────────────────────┐     HTTP      ┌─────────────────────┐
│   Client Requests   │ ────────────► │   Your API Server   │
└─────────────────────┘               │   + CVT Middleware  │
                                      └─────────────────────┘
                                               │
                                               │ Validate
                                               ▼
                                      ┌─────────────────────┐
                                      │    CVT Server       │
                                      │    + Your Schema    │
                                      └─────────────────────┘

Capabilities​

CapabilityServer Required?What It Answers
Schema compliance testsYes"Does my handler return spec-compliant responses?"
Breaking change detectionNo (CLI)"What changed between v1 and v2 of my spec?"
Consumer registryYes"Which services depend on my API?"
can-i-deployYes"Will this change break real consumers?"

Validation Approaches​

Test TypeServices RequiredSpeedPurpose
Schema ComplianceCVT onlyFastUnit test handlers against schema
Middleware ModesCVT onlyFastTest Strict/Warn/Shadow behavior
Consumer RegistryCVT onlyFastcan-i-deploy checks
HTTP IntegrationProducer + CVTMediumFull end-to-end validation
  • Schema Compliance: Test handlers directly without running a server. Fast feedback during development.
  • Middleware Modes: CVT middleware validates requests/responses in real-time. Three modes for different stages.
  • Consumer Registry: Track which endpoints consumers use. Enables can-i-deploy safety checks.
  • HTTP Integration: Real HTTP calls to running API. Tests complete stack including routing/serialization.

Schema Compliance Testing​

Schema compliance testing validates that your API handlers return responses matching your OpenAPI specification.

How It Works​

  1. Register your OpenAPI schema with CVT server
  2. Call your handler with test data
  3. Validate the response against the schema
  4. Get detailed error messages for any mismatches

Basic Example​

import { ProducerTestKit } from "@sahina/cvt-sdk/producer";

describe("Petstore API", () => {
  let testKit: ProducerTestKit;

  beforeAll(async () => {
    testKit = new ProducerTestKit({
      schemaId: "petstore",
      serverAddress: "localhost:9550",
    });
  });

  afterAll(async () => {
    testKit.close();
  });

  it("GET /pet/{petId} returns valid response", async () => {
    // Call your actual handler
    const response = await petHandler.getPet("123");

    // Validate against schema
    const result = await testKit.validateResponse({
      method: "GET",
      path: "/pet/123",
      response: {
        statusCode: 200,
        body: response,
      },
    });

    expect(result.valid).toBe(true);
    expect(result.errors).toHaveLength(0);
  });

  it("detects missing required fields", async () => {
    const result = await testKit.validateResponse({
      method: "GET",
      path: "/pet/123",
      response: {
        statusCode: 200,
        body: { id: 123 }, // missing 'name' and 'photoUrls' fields
      },
    });

    expect(result.valid).toBe(false);
    expect(result.errors[0]).toContain("name");
  });
});

Producer Middleware​

For runtime validation, add CVT middleware to your HTTP server.

How Middleware Works​

Framework Examples​

import { createExpressMiddleware } from "@sahina/cvt-sdk/producer";

app.use(
  createExpressMiddleware({
    schemaId: "petstore",
    validator,
    mode: "strict", // or 'warn' or 'shadow'
  }),
);

Path Filtering​

Exclude health checks, metrics, or other paths from validation:

createExpressMiddleware({
  schemaId: "petstore",
  validator,
  mode: "strict",
  excludePaths: ["/health", "/metrics", "/ready"],
  includePaths: ["/api/**"],
});

Validation Modes​

See Validation Modes for detailed information.

ModeRequest ViolationResponse ViolationUse Case
strictReject with 400Log errorProduction enforcement
warnLog, continueLog, continueGradual rollout
shadowSilentSilentInitial deployment
Deploy with SHADOW → Analyze metrics → Switch to WARN → Fix issues → Switch to STRICT

Consumer Registry​

Track which services depend on your API.

Listing Your Consumers​

const consumers = await validator.listConsumers("petstore", "prod");

console.log(`${consumers.length} services depend on petstore in prod`);
for (const consumer of consumers) {
  console.log(`- ${consumer.consumerId} v${consumer.consumerVersion}`);
}

Understanding Consumer Registrations​

Consumers register after their contract tests pass:

// A consumer (not you) registers like this:
await validator.registerConsumer({
  consumerId: "order-service",
  consumerVersion: "2.1.0",
  schemaId: "petstore", // Your API
  schemaVersion: "1.0.0",
  environment: "prod",
  usedEndpoints: [
    {
      method: "GET",
      path: "/pet/{petId}",
      usedFields: ["id", "name", "status"],
    },
  ],
});

This tells you:

  • order-service depends on your API
  • They use GET /pet/{petId}
  • They specifically need the id, name, and status fields

Deployment Safety (can-i-deploy)​

Before deploying a new schema version, use can-i-deploy to check if it will break any registered consumers. For complete CLI usage, SDK examples, breaking change types, and output formats, see the Breaking Changes Guide.

Test Fixture Generation​

Generate test data from your OpenAPI schema for documentation and testing.

// Generate complete request/response fixture
const fixture = await validator.generateFixture("GET", "/pet/{petId}");
console.log(fixture.request); // { method, path, headers, body }
console.log(fixture.response); // { statusCode, headers, body }

// Generate response only with schema examples
const response = await validator.generateResponse("GET", "/pet/{petId}", {
  statusCode: 200,
  useExamples: true,
});

CLI Usage​

# List all endpoints in a schema
cvt generate --schema ./openapi.json --list

# Generate fixture for an endpoint
cvt generate --schema ./openapi.json --method GET --path /pet/{petId}

# Generate request only
cvt generate --schema ./openapi.json --method POST --path /pet --output-type request

# Use examples from schema
cvt generate --schema ./openapi.json --method GET --path /pet/{petId} --use-examples

Schema Registration in CI/CD​

As the API producer, your CI/CD pipeline is responsible for registering your OpenAPI schema with the shared CVT server. This is how consumers discover your contract and how can-i-deploy knows what to check against.

Producer CI/CD Pipeline:

  Build & Test → Register Schema → Deploy
                       │
                       ▼
                  CVT Server (shared)
                       ▲
                       │
  Consumer CI/CD:  Validate interactions → Register as consumer

Register on merge to main​

# In your producer CI/CD pipeline
cvt register-schema my-api ./openapi.json \
  --version "$API_VERSION" \
  --server "$CVT_SERVER_URL" \
  --check-compatibility \
  --fail-on-breaking

Why the producer must register​

If the producer registers...If the consumer registers...
Schema always matches the real APISchema may be stale or incorrect
Single source of truthMultiple consumers may register different versions
can-i-deploy checks are meaningfulBreaking change detection is unreliable
Clear ownership and accountabilityUnclear who maintains the contract

CI/CD Integration​

For complete CI/CD pipeline examples (GitHub Actions, GitLab CI, Jenkins) including producer deployment safety checks and contract testing, see the CI/CD Integration Guide.

Best Practices​

1. Test All Response Scenarios​

Don't just test the happy path:

it("validates 404 response", async () => {
  const result = await testKit.validateResponse({
    method: "GET",
    path: "/pet/nonexistent",
    response: {
      statusCode: 404,
      body: {},
    },
  });
  expect(result.valid).toBe(true);
});

it("validates 400 response for bad request", async () => {
  const result = await testKit.validateResponse({
    method: "POST",
    path: "/pet",
    response: {
      statusCode: 400,
      body: {},
    },
  });
  expect(result.valid).toBe(true);
});

2. Run can-i-deploy in CI​

Make deployment safety checks a required gate:

cvt can-i-deploy --schema petstore --version $NEW_VERSION --env prod || exit 1

3. Use Environment-Specific Checks​

Check each environment before promoting:

# Check staging first
cvt can-i-deploy --schema petstore --version 2.0.0 --env staging

# Then production
cvt can-i-deploy --schema petstore --version 2.0.0 --env prod

4. Gradual Middleware Rollout​

Start with shadow mode, progress to strict:

// Week 1: Shadow mode - metrics only
mode: "shadow";

// Week 2: Warn mode - log violations
mode: "warn";

// Week 3: Strict mode - full enforcement
mode: "strict";

Troubleshooting​

"Schema not found" Error​

Ensure the schema is registered before running producer tests:

await validator.registerSchema("petstore", "./openapi.json");

"Path not found" Error​

Check that the path in your test matches the OpenAPI spec:

// If spec has: /pet/{petId}
// Use actual path values:
path: '/pet/123',  // NOT '/pet/{petId}'

"No consumers registered" Warning​

This is normal if you're the first to deploy or if no consumers have registered:

SAFE TO DEPLOY
No consumers registered for this schema in prod.

Next Steps​