When Contract Testing Pays for Itself

Seven posts ago I claimed that reading a small library beats using a big one through its documentation. Series over, claim due for settlement. This retrospective is the honest ledger: what Pactify — Dariusz Pawlukiewicz's open-source “very simple contract testing tool for .NET Core,” which I've now read to the last line — gets right, the complete list of what it can't do, when you should reach for PactNet instead, and the question underneath all the tooling: when does contract testing actually pay for itself?

What the small library gets right

In-memory verification is the killer feature, and Pactify made me see it. CreateFor<Startup>() replaying a consumer's real expectations through your actual middleware pipeline, at unit-test speed, with zero deployed infrastructure — that's the moment contract testing stops being a diagram and starts being a build step. Every tool in this space does something equivalent; Pactify does it in five readable lines, which is why I now understand it instead of merely invoking it.

Structural contracts are the right default. The matching engine (part 6) is only trustworthy in IgnoreContractValues = true mode — and that turned out to be a feature disguised as a limitation. “Every property my consumer reads still exists, with the right status and content type” catches the field rename, the deleted property, the route change: the actual outages. Value-matching contracts, in my experience even with richer tools, mostly catch test-data drift.

The pact as dependency inventory. The quiet win nobody advertises: after a few months of pacts, the provider team could answer “who still uses warehouse?” with a grep instead of a meeting. Deleting things became provably safe. That alone justified the setup cost.

Readable source is a real feature. Parts 2–7 exist because thirty files can be read cover to cover. The findings — the GetUninitializedObject zero-values, options travelling inside the pact, Given(...) never executing, concurrent replay, the silent publish, the two FormatException bugs in the error paths — took me an afternoon each to establish with certainty. In a tool that wraps a native core, the equivalent certainty is a support ticket.

The complete limit list

Collected from across the series, in one place, because an honest adoption decision needs the whole bill:

• Requests are method + path only. No request bodies (POST/PUT replay with null content), no request headers, no query matchers. Contracts over write-APIs degenerate to “the route exists.” (Parts 2, 6)
• No matchers at all. No “any string,” no regex, no type-matching, no array semantics. Pact spec's matchingRules — the feature that makes value assertions survive real data — has no counterpart. (Parts 3, 6)
• Body matching is top-level-only; nested objects are presence-checks at best and reference-equality failures at worst. (Part 6)
• Given is documentation. No provider-state hooks; the provider seeds one global state satisfying every interaction simultaneously (they replay concurrently). (Part 5)
• Document-wide dials. Options and the endpoint template apply per-pact, not per-interaction. One lenient interaction makes the whole document lenient. (Parts 3, 5)
• Error paths with two format-string bugs that turn value-mismatch diagnostics into FormatExceptions; header names compared case-sensitively regardless of IgnoreCasing. (Part 6)
• The HTTP publisher never checks the broker's response — a failed publish passes your build. (Part 4)
• Pact-v1-adjacent, not Pact-compliant. Broker-storable, but no matchingRules/metadata, plus a nonstandard options block; assume Pactify-to-Pactify only. Broker auth is a custom Pact-Requester header, not the Authorization schemes real brokers use. (Parts 3, 4)
• A 2.x-era footprint. Startup-class convention, TestHost 2.2, IRouter-based endpoint, Newtonsoft throughout — workable, but visibly dated against minimal hosting and WebApplicationFactory. (Parts 5, 7)

That list reads brutally in bulk, so let me re-balance it: none of these limits touched the outage that opened this series. The renamed-field regression — the representative disaster of the genre — is caught completely by structural matching over GET endpoints, which is precisely the slice Pactify implements well. The limits define a boundary, not a verdict.

Pactify or PactNet?

My decision rule, having run one and read the other's problem space through it:

Choose Pactify when the seams you're protecting are read-shaped internal APIs between a handful of .NET teams (or one team's monorepo of services), you want the whole team to understand the mechanism, and your contracts are structural. First-touch cost is an afternoon; the pact-in-git workflow (part 4) needs no infrastructure at all. It is also, hands down, the best teaching codebase for the concept — I'd assign parts of it as reading before letting anyone near a broker.

Choose PactNet when you need request bodies and headers in the contract, matchers instead of literals, executable provider states, polyglot consumers (the pact format is the interop layer — Pactify's dialect isn't), spec v3/v4 compliance, or broker workflows with deployment gates. That's not a feature-envy list; each item maps to one of the hard limits above. PactNet buys them by wrapping the Pact foundation's Rust core — the price is that your judgment engine becomes a native binary you'll never read. After this series I consider that a real price, worth paying knowingly rather than by default.

The upgrade path is gentler than it looks: everything organizational — consumer tests as first-class CI citizens, pact publication steps, provider verification gates, the habit of treating the seam as tested — transfers wholesale. You're swapping the recorder and the judge, not the discipline.

The broker maturity ladder

Where teams actually get stuck is never the library; it's the workflow. The rungs, as I've climbed them:

1. Pacts in the repo. Monorepo folder, git as history, PR review as the change-control conversation. Start here; stay here longer than feels sophisticated.
2. Pacts as pipeline artifacts. Cross-repo, consumer publishes, provider downloads. Works until the first “which version did we verify?” incident.
3. A broker. Versioned pacts, a UI both teams bookmark, verification results recorded centrally. This is where “the contract” becomes a shared noun in planning meetings — the cultural payload matters more than the Docker container.
4. Deployment gates. The full Pact-ecosystem endgame — webhooks triggering provider verification on pact change, can-i-deploy blocking releases until the matrix is green. Pactify has no native rungs here; this level is PactNet-and-friends territory.

Each rung costs an order of magnitude more organizational buy-in than the last, and delivers the same multiple in value — if you have the problem the rung solves. Climbing ahead of your problems is how contract testing initiatives die: a broker nobody consults is a second integration environment with better branding.

When it pays for itself

The tooling was never the expensive part. The consumer-driven contract is an organizational instrument — it moves the veto over provider changes into the consumers' recorded expectations, and pays off exactly where that inversion matters:

• Multiple teams, independent deploy cadences. The seam breaks precisely when coordination is expensive. This is the home fixture.
• Churning internal APIs. Contracts shine while shapes are still being negotiated; they're a rename-detector, and renames happen early and often.
• A shared integration environment everyone hates. Not because contracts replace E2E — they don't — but because they drain the cheap failure classes out of it (shape drift, dead routes), leaving the environment to catch what genuinely needs everything running: emergent behaviour, data flow, timing.

And the honest negative space — where I'd now decline to introduce it: one team owning both sides of every seam (your integration tests already share types; the contract adds ceremony, not information), a public API with unknown consumers (consumer-driven requires enumerable consumers — you want OpenAPI, versioning policy and deprecation windows instead), and genuinely stable interfaces, where a pact is a smoke detector in a stone house.

The most durable change wasn't any pipeline. It was that “will this break checkout?” became a question with a mechanical answer — asked of a JSON document in CI instead of a person in a meeting. Once a team has tasted that, they keep the discipline even when they swap the tool. Which is, I think, the strongest endorsement available for starting with a thirty-file library you can read in an afternoon: Pactify is small enough to fully understand, and consumer-driven contract testing is exactly the kind of idea you want to understand fully before you industrialize it. Credit to its author for shipping the sketch that teaches the discipline — and if the sketch's rough edges bothered you across this series, remember they're all one afternoon and a pull request away. That's what open source is for.