Skip to main content

Contract Validator Toolkit (CVT)

Historical Document

This PRD captures the original vision and requirements that guided CVT development. Some implementation details may have evolved since this document was written. For current information, see:

1. Overview​

This document defines the product, architectural, and implementation requirements for the Contract Validator Toolkit (CVT) — a consumer- and producer-based contract validation platform for OpenAPI v2 and v3 specifications. CVT is built using Go, exposes a gRPC-first interface, and runs entirely via Docker

2. Definitions​

Contract testing validates API interactions against a published contract. It supports two primary modes:

TermDefinition
Consumer-based Contract TestingValidation initiated by an API consumer to ensure that requests and responses conform to a published OpenAPI contract.
Producer-based Contract TestingValidation performed by an API provider to ensure their implementation conforms to the contract.

CVT supports both consumer and producer contract testing​

3. Problem Statement​

Developers and API consumers need a consistent, language-agnostic way to validate API requests and responses against OpenAPI contracts. In practice, teams often rely on embedding or wrapping existing validation libraries (for example, a mature Java-based contract validator) inside their test environment. Two common integration approaches create markedly different operational trade-offs:

  • Bridge / Host-bound approach: Integrates a validator directly, leading to brittle runtime dependencies, host environment coupling, and "works on my machine" inconsistencies across dev, CI, and production.
  • Docker-sandboxed CVT approach: Runs the validator in an isolated container via a gRPC interface, ensuring reproducible and consistent validation across all environments by encapsulating dependencies. The Contract Validator Toolkit (CVT) is a proposal for the Docker-sandboxed approach. CVT centralizes contract validation in a lightweight, containerized service that provides a consistent gRPC API. The following sections explain the operational differences with examples, and visualize the two deployment/runtime models. Key problems seen with bridge/host-bound integration:
  • Host dependency drift: different runtime versions or local dependency versions cause validation logic to behave differently across machines.
  • Fragile bridging: adapters and native bindings often fail silently or produce obscure errors when ABI or OS expectations differ.
  • CI inconsistency: containers or runner images that lack required native libraries or runtime dependencies break tests unpredictably.
  • Security surface: running third-party validators in-process increases attack surface and makes resource isolation difficult.
  • Version governance: coordinating validator library upgrades across multiple repositories becomes a release engineering burden. How a Docker-based CVT solves these problems:
  • Encapsulation: validator and its transitive dependencies are bundled into an immutable image.
  • Reproducibility: identical behavior across developer laptops, CI runners, and centralized instances.
  • Isolation: resource limits, user namespaces, and network restrictions protect the host.
  • Central governance: a shared CVT instance can be managed, monitored, and upgraded independently from consumer codebases.

The subsections below visualize each approach and highlight where failures commonly occur.

3.1 Bridge / Host-bound Validator (Brittle)​

This approach embeds a language-specific validator directly into the test environment, sharing the host's runtime and native libraries. This tight coupling often leads to brittle runtime dependencies and inconsistencies across different environments (e.g., "works on my machine" failures, CI flakiness).

This approach encapsulates the validator and its dependencies within a container image, exposing it via a stable gRPC interface. Test runtime SDKs communicate with the container over the network, ensuring reproducible validation behavior across all environments (dev, CI, central) and simplifying upgrades.

4. Goals and Objectives​

  • Focus on consumer- and producer-based contract testing for OpenAPI v2 and v3.
  • SDKs are not test frameworks — users continue to use existing testing libraries like Jest, Pytest, or JUnit.
  • SDKs handle communication and orchestration only: gRPC protocol handling, configuration, authentication, and result retrieval.
  • Maintain uniform and simple SDK functionality across Node, Python, Go, and Java.
  • Deliver Docker-only deployments suitable for local, central, or CI use.
  • Lightweight and performant: Go-based server for minimal resource usage and fast startup.

5. System Architecture Summary​

The system follows a strict Client-Server architecture.

Components​

  1. Client – The consumer-side SDKs and test runners (Node, Python, Go, Java). Handles gRPC protocol abstraction and test execution.
  2. Server – The CVT Container (Go-based), handling validation logic with kin-openapi and in-memory schema caching.

5.1 Project Structure (Monorepo)​

The project adopts a Monorepo structure to ensure strict synchronization between the API definitions (Protobuf/OpenAPI) and the implementation (Server + SDKs).

/
├── api/                        # SINGLE SOURCE OF TRUTH
│   ├── protos/                 # gRPC definitions (cvt.proto)
│   └── openapi/                # OpenAPI specs
├── cmd/cvt/                    # CLI entry point (main.go)
├── server/                     # Go Server Implementation
│   ├── cvtservice/             # Core service logic
│   │   ├── validator_service.go  # Core validation service
│   │   ├── compatibility_engine.go # Breaking change detection
│   │   ├── cache.go            # Schema caching (Ristretto)
│   │   ├── health.go           # Health check service
│   │   ├── logger.go           # Structured logging (Zap)
│   │   └── ...
│   ├── storage/                # Persistence backends (sqlite, postgres, memory)
│   ├── pb/                     # Generated protobuf code
│   └── Dockerfile              # Multi-stage build
├── pkg/cvt/                    # Embeddable Go library (CLI lite mode)
├── sdks/                       # Client Libraries
│   ├── java/
│   ├── node/
│   ├── python/
│   └── go/
├── config/                     # Configuration files
├── observability/              # Prometheus/Grafana configuration
└── tools/                      # Shared build/codegen scripts

6. Runtime and Execution Models​

ModeDescriptionPersistence
Local Developer ModeRun CVT locally via Docker Compose for iterative contract validation.File-based or SQLite
Centralized ModeShared internal instance for team-wide validation.PostgreSQL
Ephemeral CI ModeCVT runs temporarily in CI and terminates after tests.File-based or in-memory

Run locally with:

docker compose up -d

7. Functional Requirements​

7.1 Schema Registration​

  • Register OpenAPI v2/v3 schemas.
  • Convert v2 → v3 at load time via kin-openapi's openapi2conv.
  • Assign schema ID, hash, and version.

7.2 Validation Runs​

  • Submit request/response payloads for validation.
  • Return structured results (pass/fail + error details).

7.3 SDK Behavior​

The SDK provides a simple, high-level API to validate API interactions against the contract. It is designed to be used alongside any test framework (Jest, Mocha, Pytest, etc.).

1. Instantiation Initialize the validator with a schema source (local file or remote ID).

const validator = new ContractValidator({
  schema: "./openapi.yaml", // Or a URL/ID for remote
});

2. Validation The validate method takes two arguments: the Request (what was sent) and the Response (what was received).

const result = await validator.validate(
  // Request
  {
    method: "POST",
    path: "/users",
    body: { name: "Alice" },
  },
  // Response
  {
    statusCode: 201,
    body: { id: 1, name: "Alice" },
  },
);

3. Assertion The result object is a simple boolean/error structure, making it compatible with any assertion library.

// Jest
expect(result.valid).toBe(true);

// Chai
assert.isTrue(result.valid);

// Native Node.js
assert.ok(result.valid, result.errors?.join(", "));

4. Fluent API (Alternative) For developers who prefer a chained builder pattern, the SDK offers a fluent interface:

const result = await validator
  .interaction()
  .request({ method: "POST", path: "/users", body: { name: "Alice" } })
  .response({ statusCode: 201, body: { id: 1 } })
  .validate();

Python​

from cvt_sdk import ContractValidator

validator = ContractValidator(schema="./openapi.yaml")

result = validator.validate(
    request={"method": "POST", "path": "/users", "body": {"name": "Alice"}},
    response={"status_code": 201, "body": {"id": 1}}
)

assert result.valid, f"Contract failed: {result.errors}"

Java​

import io.github.sahina.sdk.ContractValidator;
import static org.junit.jupiter.api.Assertions.assertTrue;

ContractValidator validator = new ContractValidator(new Config().schema("./openapi.yaml"));

ValidationResult result = validator.validate(
    Request.builder().method("POST").path("/users").body("{\"name\": \"Alice\"}").build(),
    Response.builder().statusCode(201).body("{\"id\": 1}").build()
);

assertTrue(result.isValid(), "Contract violation: " + result.getErrors());

Go​

import (
    "testing"
    "github.com/sahina/cvt/sdks/go/cvt"
    "github.com/stretchr/testify/assert"
)

func TestContract(t *testing.T) {
    validator := cvt.NewValidator(cvt.Config{Schema: "./openapi.yaml"})

    result, _ := validator.Validate(
        cvt.Request{Method: "POST", Path: "/users", Body: map[string]any{"name": "Alice"}},
        cvt.Response{StatusCode: 201, Body: map[string]any{"id": 1}},
    )

    assert.True(t, result.Valid, "Contract violation: %v", result.Errors)
}

7.4 Compatibility Engine​

  • Compare schema versions and flag breaking/non-breaking changes via CompareSchemas RPC.
  • Detects endpoint removal, required field additions, type changes, parameter additions, response schema changes, and enum value removal.

8. Typical Use Cases and Flow​

8.1 Local Developer Workflow​

8.2 CI Pipeline Workflow​

9. gRPC Service Definition (Pseudocode)​

service ContractValidator {
  // Schema & Validation
  rpc RegisterSchema(RegisterSchemaRequest) returns (RegisterSchemaResponse);
  rpc ValidateInteraction(InteractionRequest) returns (ValidationResult);
  rpc GetSchema(GetSchemaRequest) returns (GetSchemaResponse);
  rpc ListSchemas(ListSchemasRequest) returns (ListSchemasResponse);
  rpc CompareSchemas(CompareSchemasRequest) returns (CompareSchemasResponse);
  rpc GenerateFixture(GenerateFixtureRequest) returns (GenerateFixtureResponse);
  rpc ListEndpoints(ListEndpointsRequest) returns (ListEndpointsResponse);
  // Producer Testing
  rpc ValidateProducerResponse(ValidateProducerRequest) returns (ValidationResult);
  // Consumer Registry
  rpc RegisterConsumer(RegisterConsumerRequest) returns (RegisterConsumerResponse);
  rpc ListConsumers(ListConsumersRequest) returns (ListConsumersResponse);
  rpc DeregisterConsumer(DeregisterConsumerRequest) returns (DeregisterConsumerResponse);
  // Deployment Safety
  rpc CanIDeploy(CanIDeployRequest) returns (CanIDeployResponse);
}

Note: See API Reference for the full proto definition with all message types. The pseudocode above is abbreviated.

10. Validation Engine Design​

Core engine uses Go ecosystem libraries (specifically kin-openapi) for robust OpenAPI v2/v3 validation.

Key Libraries:

  • kin-openapi (v0.134.0): OpenAPI 3.0/3.1 and Swagger 2.0 parsing and validation
  • Ristretto (v0.2.0): High-performance schema caching (LRU, 1000 schemas max, 24h TTL)
  • Zap (v1.27.1): Structured logging

Core Implementation:

// ValidatorService implements the gRPC ContractValidator service
type ValidatorService struct {
    cache *SchemaCache // Ristretto cache for compiled validators
}

func (s *ValidatorService) RegisterSchema(ctx context.Context, req *pb.RegisterSchemaRequest) (*pb.RegisterSchemaResponse, error) {
    // Parse and validate OpenAPI schema
    loader := openapi3.NewLoader()
    doc, err := loader.LoadFromData([]byte(req.SchemaContent))
    if err != nil {
        return &pb.RegisterSchemaResponse{Success: false, Message: err.Error()}, nil
    }

    // Validate the OpenAPI document
    if err := doc.Validate(loader.Context); err != nil {
        return &pb.RegisterSchemaResponse{Success: false, Message: err.Error()}, nil
    }

    // Store in cache
    s.cache.Set(req.SchemaId, doc)
    return &pb.RegisterSchemaResponse{Success: true}, nil
}

func (s *ValidatorService) ValidateInteraction(ctx context.Context, req *pb.InteractionRequest) (*pb.ValidationResult, error) {
    // Retrieve compiled validator from cache
    doc, found := s.cache.Get(req.SchemaId)
    if !found {
        return &pb.ValidationResult{Valid: false, Errors: []string{"Schema not found"}}, nil
    }

    // Create router and find matching operation
    router, _ := gorillamux.NewRouter(doc)
    route, pathParams, err := router.FindRoute(httpReq)
    if err != nil {
        return &pb.ValidationResult{Valid: false, Errors: []string{err.Error()}}, nil
    }

    // Validate request and response
    if err := openapi3filter.ValidateRequest(ctx, requestInput); err != nil {
        return &pb.ValidationResult{Valid: false, Errors: []string{err.Error()}}, nil
    }
    if err := openapi3filter.ValidateResponse(ctx, responseInput); err != nil {
        return &pb.ValidationResult{Valid: false, Errors: []string{err.Error()}}, nil
    }

    return &pb.ValidationResult{Valid: true}, nil
}

Architecture Benefits:

  • Performance: 5000+ validations/second, <1 second startup time
  • Memory Efficiency: ~50-100MB base memory usage
  • Container Size: ~30-40MB Docker image (vs ~200MB+ for Java)
  • Binary Distribution: Single static binary (~19MB), no runtime dependencies

11. SDK Deliverables​

SDKLanguagePurposeStatusDistribution
Node.jsTypeScriptPrimary reference implementation (gRPC)✅ Complete@sahina/cvt-sdk
PythonPythonPytest/unittest compatible (gRPC)✅ Completecvt-sdk
GoGoGo test integration (gRPC)✅ Completepkg.go.dev
JavaJavaJUnit integration (gRPC)✅ CompleteMaven Central

Note: Current implementation is gRPC-only. REST fallback may be added in future phases if needed.

SDKs are pure validator clients. They do not execute HTTP requests; they only transport request/response data to the CVT container for validation.

12. Non-functional Requirements​

CategoryRequirement
Performance5000+ validations/sec, <1s startup time, <10ms schema registration
Reliability99.9% uptime (central mode)
SecurityAPI key auth, TLS/mTLS, sandboxed validator, non-root container
ObservabilityStructured logs (Zap), gRPC health checks, metrics
CompatibilityOpenAPI v2 (Swagger) and v3.0/v3.1
Resource Usage~50-100MB memory, ~30-40MB container image

13. Deployment​

  • Docker-only deployment using Docker Compose
  • Supports local development, centralized, and CI usage Example CI integration:
steps:
  - name: Start CVT
    run: docker compose up -d
  - name: Run Contract Tests
    run: npm test
  - name: Stop CVT
    run: docker compose down

14. Phased Development Plan​

Initial Phase – MVP (Completed)​

  • ✅ Client-Server architecture
  • ✅ All SDKs (Node, Python, Go, Java)
  • ✅ Core Validation Logic with kin-openapi
  • ✅ Docker and Docker Compose deployment
  • ✅ Go server implementation (migrated from Java)

Migration to Go (November 2025):

  • Replaced Java server with Go implementation for better performance and resource efficiency
  • Zero breaking changes: 100% API compatibility maintained
  • Library: Using kin-openapi for OpenAPI 2.0/3.0/3.1 validation
  • Benefits: 5x smaller images, 3-5x faster startup, 5x less memory usage
  • All existing SDKs work without modification

Phase P1 – Persistence & Observability (Completed)​

  • ✅ Flexible persistence options: SQLite and PostgreSQL storage backends
  • ✅ Observability stack: Prometheus metrics, Grafana dashboards
  • ✅ TLS/mTLS and API key authentication
  • ✅ Producer testing (ValidateProducerResponse)
  • ✅ Consumer registry (RegisterConsumer, ListConsumers, DeregisterConsumer)
  • ✅ Deployment safety (CanIDeploy)
  • ✅ Breaking change detection (CompareSchemas)
  • ✅ Test fixture generation (GenerateFixture)
  • 🔲 Admin UI / Dashboard for schema management and visualization

Phase P2 – Multi-component Scalability​

  • Scalability enhancements
  • Multi-component architecture support

15. Risks and Mitigations​

15.1 Key Risks​

RiskImpactMitigationOwner
OpenAPI v2 QuirksHigh• Explicit conversion pipeline (lint → convert → validate)
• Surface warnings in SDK
• "Dry-run" conversion endpoint		Schema Team
False Positives in DiffMedium• Deterministic diff rules
• Machine-readable changelogs
• Human-in-the-loop review workflow		Compatibility Team
Resource ConstraintsLow• Go's efficient memory model (~50-100MB base)
• Small container images (~30-40MB)
• Fast startup times (<1s)
• Hardened runtime (non-root)		Ops / Infra
PerformanceLow• Ristretto LRU cache for compiled validators (1000 max, 24h TTL)
• Go's efficient concurrency (goroutines)
• 5000+ validations/sec baseline
• Horizontal scaling if needed		Core Engine Team

15.2 Cross-cutting Risks​

  • Security: Scan uploads for malicious constructs (ReDoS, infinite recursion). Run unprivileged.
  • Governance: Require ownership metadata. Audit logs for all changes.
  • Observability: Return structured error codes. Store original & converted specs.
  • Compatibility: Strict API versioning. Test matrix for gRPC & SDKs across languages.
  • Compliance: Configurable retention policies. Encryption-at-rest.

15.3 Next Steps​

  • Implement conversion-preview endpoint.

  • Define diff classification rules.

  • Benchmark validation throughput.

Risk: Data retention and compliance for stored specs/results​

  • Mitigation:
    • Provide configurable retention policies for runs and specs, deletion/purge APIs, and encryption-at-rest for persisted stores.
  • Owner: Platform / Compliance

15.4 High-priority next steps (actionable)​

  • Convert each short risk line into the expanded format above and assign owners in your project tracker.
  • Implement conversion-preview and surface conversionWarnings in registration APIs.
  • Define and publish diff classification rules and a curated test set for validating the compatibility engine.
  • Create a benchmark plan for validation throughput and start collecting baseline metrics in a staging environment.

16. Future Enhancements​

16.1 Admin Dashboard​

Goal: GUI for schema management, run visualization, and reporting. Stack: Next.js App Router + ShadCN.

16.2 Mocking & Simulation​

Goal: Serve synthetic, schema-compliant responses for E2E testing without live backends. Supports Faker.js templates for realistic data.

16.3 Plugin Architecture​

Goal: Support non-OpenAPI formats (AsyncAPI, GraphQL, Protobuf) via dynamic plugins.

16.4 Governance & Intelligence​

  • Policy Engine: Enforce rules (e.g., "No breaking changes") via YAML policies.

  • Federation: Secure, encrypted schema sharing across teams with RBAC.

  • AI Assistant: Auto-generate schemas, suggest fixes, and summarize changelogs.

17. Appendix​

  • UX Study for Ephemeral Environments (internal Mural board — access restricted)