Replaying the Contract Against a Real API

Everything in this series so far has been promises. The consumer recorded expectations it never checked; the pact travelled to a broker that verified nothing. The provider side is where consumer-driven contract testing finally cashes out: take each recorded interaction, fire the real request at the real API, and compare the real response to what the consumer wrote down.

The remarkable thing about how Pactify (the small open-source .NET contract testing library from DevMentors that this series reads cover to cover) does this is what's absent: no deployed environment, no localhost port, no network socket at all. This post walks the verification flow end to end — including two behaviours you won't guess from the API surface, one of which quietly redefines what a keyword you've already used actually means.

Your whole API, hosted inside a unit test

The provider's entire test:

[Fact]
public async Task Inventory_Honours_Checkouts_Contract()
{
    await PactVerifier
        .CreateFor<Startup>()
        .Between("checkout", "inventory")
        .RetrievedFromFile("../../../../../pacts")
        .VerifyAsync();
}

The magic is in CreateFor<TStartup>, and it's five lines:

// src/Pactify/PactVerifier.cs
public static IPactVerifier CreateFor<TStartup>() where TStartup : class
{
    var testServer = new TestServer(new WebHostBuilder().UseStartup<TStartup>());
    var httpClient = testServer.CreateClient();
    return Create(httpClient);
}

TestServer, from Microsoft.AspNetCore.TestHost, boots your actual Startup — real DI registrations, real middleware pipeline, real routing, real serializers — but replaces the network transport with an in-memory channel. The HttpClient it hands back looks ordinary, yet its requests never touch a socket; they're delivered straight into your application's request pipeline as objects. Verification runs at unit-test speed, in CI, with zero infrastructure, against the code exactly as it will run in production minus Kestrel.

This is the same in-memory hosting trick that powers WebApplicationFactory in modern ASP.NET Core integration testing, and seeing it here — as the foundation of a contract testing tool — is what made the whole category click for me. A contract test is just an integration test whose requests were written by another team.

And because everything downstream only needs an HttpClient, the escape hatch is trivial: PactVerifier.Create(myHttpClient) pointed at a deployed staging URL turns the same pact into a smoke test against real infrastructure. Same contract, two altitudes. (For the in-memory route, note that Pactify targets the 2.x-era TestHost and the Startup-class convention — a real consideration for minimal-hosting apps, which I'll weigh in the retrospective.)

The replay loop, and a & you don't see every day

Here's the heart of the verifier:

// src/Pactify/PactVerifier.cs
public async Task VerifyAsync()
{
    var definition = await _retriever.RetrieveAsync();
    var verifier = new HttpInteractionVerifier(_httpClient);

    var resultTasks = definition.Interactions
        .Select(c => verifier.VerifyAsync(c, definition.Options, _pathTemplateObject))
        .ToList();

    await Task.WhenAll(resultTasks);

    var result = resultTasks
        .Select(t => t.Result)
        .Aggregate((c, next) => c & next);

    if (result.IsSuccessful)
    {
        return;
    }

    throw new PactifyException(string.Join(Environment.NewLine, result.Errors));
}

Four steps: retrieve the pact (file or broker — the verifier neither knows nor cares), replay every interaction, merge the results, and either return quietly or throw one exception carrying every failure.

Two details deserve a closer look.

That c & next works because PactVerificationResult overloads the & operator — combining two results concatenates their error lists, and IsSuccessful is simply “no errors.” It's a small flourish (an Aggregate over a Combine method would read the same), but it gives the verifier its best property: failures don't short-circuit. Every interaction is checked, every violation is collected, and the final PactifyException message is the complete bill — “status was 404 not 200” and “header missing” and “property available absent” — in one run. When a provider refactor breaks four interactions, you learn about all four immediately, not one per CI round-trip.

The second detail is easy to read past: Select(...VerifyAsync...) starts all interactions before Task.WhenAll awaits any of them. Verification is concurrent. Against an in-memory TestServer that's usually invisible — until it isn't. Your interactions cannot depend on each other (“the POST runs before the GET” is not a promise anyone made), and any state they touch must be valid for all of them simultaneously. Which raises an obvious question: who sets up that state?

Given(...) doesn't do what you think

In Pact.io proper, provider states are executable: the verifier announces “this interaction requires A SKU exists with stock on hand,” and the provider registers a setup handler that seeds exactly that data before the request is replayed. It's the mechanism that lets contract tests exercise data-dependent endpoints.

I went looking for Pactify's equivalent and found this: the State property is written into the pact as provider_state, deserialized back on the provider side… and never read again. Search the verifier — nothing consumes it. Given(...) is documentation. Sincere, useful documentation that names the world the consumer assumed — but no code will ever arrange that world for you.

The practical consequence: the provider seeds state globally, in the test host itself, and it must satisfy every interaction in the pact at once (remember, they replay concurrently). Pactify's own test suite shows the intended pattern — its Startup maps the exact route the pact will request and returns a canned model. In a real service you keep your production Startup and swap the data layer underneath it:

public class ContractTestStartup : Startup   // inherit the real pipeline
{
    public override void ConfigureServices(IServiceCollection services)
    {
        base.ConfigureServices(services);
        // replace the repository with one pre-seeded to satisfy
        // every provider_state in the pact, all at once
        services.AddSingleton<IStockRepository>(new InMemoryStockRepository(
            new StockLevel(new Guid("9b5055dd-5750-4c65-ab4c-9a785a9b7ef4"), "MAIN", 12)));
    }
}

Then PactVerifier.CreateFor<ContractTestStartup>(). It works well — but it centralizes state setup in a way that per-interaction handlers don't, and as pacts grow, that seeded repository becomes a little museum of every consumer assumption ever recorded. Budget for tending it.

Templated paths: SmartFormat at the seam

One problem remains. The consumer recorded api/stock/ABC-123 — a literal. What if the provider's seeded data uses a different identifier? What if the consumer doesn't want to hard-code one at all?

Pactify's answer is endpoint templating. The consumer records the path with a placeholder:

.WithPath("api/stock/{Sku}")

and the provider, at verification time, supplies an object whose properties fill the holes:

await PactVerifier
    .CreateFor<ContractTestStartup>()
    .UseEndpointTemplate(new StockLevelModel
    {
        Sku = new Guid("9b5055dd-5750-4c65-ab4c-9a785a9b7ef4")
    })
    .Between("checkout", "inventory")
    .RetrievedFromFile("../../../../../pacts")
    .VerifyAsync();

The implementation is one line in the interaction verifier, delegated to the SmartFormat library:

// src/Pactify/Verifiers/HttpInteractionVerifier.cs
var requestPath = templateObject is null
    ? definition.Request.Path
    : Smart.Format(definition.Request.Path, templateObject);

Smart.Format is string.Format with named, property-resolving placeholders — {Sku} binds to the template object's Sku property. It's a genuinely elegant division of authority: the consumer owns the URL's shape, the provider owns the identifier values, which is exactly who knows what. The provider seeds a row, passes the same key into the template, and the two meet in the middle.

Its limits are as clear as its elegance: one template object for the entire pact (it's passed to every interaction identically), so two interactions needing different values for {Sku} can't be expressed — the same document-wide granularity we hit with options in part 3. And since a placeholder-bearing path is meaningless to any other tooling, templating quietly deepens the Pactify-to-Pactify coupling of your pacts.

Trust, then verify the verifier

So: the pact retrieved, every interaction replayed concurrently through an in-memory copy of the real API, results merged by an overloaded operator, failures reported in full. What I haven't shown is the judgment call at the centre — how each replayed response is compared against the recorded expectation. That comparison engine is where Pactify's simplicity bites hardest: header names that ignore your casing option, nested objects that can never match, and two error paths that crash instead of reporting. That's the next post, and it's the one where reading the source pays its whole ticket price.