Two Gateways and a Nervous System

Whether POST /orders becomes an HTTP call or a RabbitMQ message depends on which docker-compose file you happened to start. In the Pacco estate, compose/services.yml — the file that pulls published images from Docker Hub — sets the gateway's config to ntrada-async.docker.yml, which publishes mutating requests straight onto the bus and answers 202. compose/services-local.yml — the file that builds the same services from your local clones — sets ntrada.docker, which proxies them downstream over HTTP and answers whatever the service answers. Same estate, same YAML gateway, two different distributed systems, and nothing in either file mentions that they disagree.

That is not a bug you find by reading a service. It is a finding you can only make by reading the estate — the compose files, the process manifests, the clone scripts, the config that stitches thirteen repositories into one running system. This series is that read.

Pacco is the open-source parcel-delivery estate by DevMentors — Piotr Gankiewicz and Dariusz Pawlukiewicz — published at github.com/devmentors under MIT: ten backend microservices, a saga-driving client, and two API gateways, spread across thirteen sibling repositories over the authors' Convey framework. To be clear up front: I did not write Pacco. I am a source-reader working through a well-known teaching estate, and every claim in this series is grounded in the actual repositories, cited by file path. Where the implementation is the story I will quote it, attributed; everything else is fresh example code. A companion series, The Order That Choreographs Itself, reads the domain that lives inside these services; this one deliberately stays outside them — at the edges, the wiring and the platform.

Thirteen repositories, one map

The estate splits cleanly into five groups, and the split is the series' table of contents:

The umbrella repo, Pacco, contains no service code at all. Its Pacco.sln references 41 .csproj files through relative ..\ paths into the twelve sibling repositories — a solution-of-solutions that only compiles if you cloned everything into the same parent directory, which is exactly what scripts/git-clone.sh does. The umbrella is a repo whose only artefact is the shape of your working directory. Everything about running Pacco — which containers start, which config the gateway loads, which services even exist at runtime — is decided here, one repository away from any of the code it governs.

And the umbrella has opinions it doesn't state. Its README lists twelve repositories to clone; thirteen exist. The one it forgets, Pacco.APIGateway.Ocelot, is in the clone script's REPOSITORIES array — cloneable, buildable, pushed to Docker Hub by its own CI, and wired into no compose file and no process manifest. The estate carries a second front door that no run configuration ever opens. That half-adopted twin gets a full part of this series, because the comparison between the two gateways is one of the best architecture lessons in the estate.

A platform wearing production clothes

Bring the estate up and you get a genuinely impressive skyline: compose/infrastructure.yml starts ten infrastructure containers on a shared pacco-network — Consul for service discovery, Fabio load-balancing over it, RabbitMQ built as a custom image with the Prometheus plugin enabled, MongoDB, Redis, Vault, Jaeger, Prometheus, Grafana and Seq. Every service registers with Consul, reports traces to Jaeger at 100 % sampling, and exposes /metrics for a Prometheus that scrapes all eleven application containers by DNS name every five seconds. It looks like a CNCF reference diagram.

Read the same file more slowly and the production clothes come off. Of the ten containers, only Mongo and Redis have volumes; the volume blocks for Grafana, Prometheus, RabbitMQ and Seq are all present — and commented out. Dashboards, time-series data, queue state and logs evaporate on docker-compose down. Vault runs as a dev-mode container whose root token is set inline in the compose file; every service's local configuration authenticates to it with that same placeholder token, and every docker configuration disables Vault entirely. RabbitMQ runs on its default account, Mongo runs unauthenticated (the credentials block is committed, commented out), and Seq's API key is the literal word you would guess. I name these as findings rather than reproduce them as config, and it is only fair to add: this is a teaching estate, and dev-mode defaults are a defensible choice for one. But the pattern — a secrets manager present in the diagram and disabled in the deployment, observability with no memory — is exactly what this series exists to catch, because I have seen both survive the copy-paste into estates that were not samples.

The genuinely delightful counterpoint is docker-images.txt, a 460-line plain-text run-book in the umbrella that documents, among other things, a full production Vault flow: operator init, unseal, policies, a database secrets engine issuing dynamic per-service MongoDB credentials with one-hour TTLs, and a PKI engine issuing per-service certificates. None of it is wired to anything. The estate's most mature security thinking lives in a text file that no container reads.

Three ways to run one estate

Pacco can be started three ways, and they do not agree with each other. Docker compose is the documented path: infrastructure first, then services, joined across files by an external network. PM2 — the Node.js process manager — is the second: the umbrella's root-level services.yml and prod-services.yml are PM2 app manifests that run each service as a raw dotnet process. And plain dotnet run per repository is the third.

The PM2 manifests list ten apps. The compose file lists eleven services. The missing one is ordermaker-service — the Chronicle saga client that actually drives an order end-to-end. Under PM2, the order-making saga silently never runs, and nothing warns you; the estate simply behaves as if that feature does not exist. Add the gateway-config disagreement from the cold open and a pattern emerges that will recur throughout this series: in a polyrepo estate, the run configurations are unversioned contracts, and they drift exactly like the copied code does.

The nervous system

The service that gives this series half its title is Pacco.Services.Operations. It is the smallest interesting service I have read in years. It subscribes to the entire estate's message traffic — every command, every event, every rejection, across eight exchanges — while owning zero contract classes. It does this by reading a hand-maintained JSON registry of the estate's messages and emitting CLR types for them at runtime with Reflection.Emit, then funnelling all of them into three generic handlers backed by a five-minute Redis state machine and a per-user SignalR push. It is how a client that received a bare 202 from the gateway ever learns what became of its request.

Operations is the estate's answer to three hard questions at once — shared contracts, async-REST feedback, and real-time push — and each answer has a sharp edge worth a full article. It gets three.

Where the series goes

1. Two Gateways and a Nervous System — this post.
2. Subscribing to Everything Without Referencing Anything — Operations' runtime-emitted message types.
3. One YAML Flag Turns REST Into a Message — ntrada.yml versus ntrada-async.yml.
4. A Gateway in YAML and Its Twin in Code — the Ntrada versus Ocelot bake-off.
5. The 202 That Names the Future — the async-REST feedback loop end-to-end.
6. Two Push Channels, Two Security Models — SignalR versus the gRPC stream.
7. The Correlation Spine — one JSON blob from edge to bus to browser.
8. Security Theatre and Security Reality — the posture, measured against the run-book.
9. Identity in Fifteen Files — JWT issuance done small, with one real bug.
10. Thirteen Repos and a Floating Version — polyrepo mechanics and the 0.4.* wildcard.
11. The Repo That Runs the Other Twelve — the umbrella as a genre.
12. Observability With Amnesia — full instrumentation on volume-less containers.
13. The Grammar of the Estate — the naming system as a readable language.
14. The Message Map and the Wrong Exchange — who publishes what, and the bug in the map.
15. Cartography of a Successor — the retrospective, including how Pacco evolved past DShop.

A note on era before we start: Pacco targets netcoreapp3.1 — roughly 2020, end-of-life since December 2022 — and every Convey and Ntrada package floats at a 0.4.* wildcard. The Travis CI badges are dead. I will date findings honestly and critique the ideas, not the calendar year.

If you have ever inherited a system that was really twelve systems and a directory convention — or drawn a beautiful platform diagram and quietly known the volumes were commented out — the next fourteen parts are for you. We start with the estate's best trick: subscribing to everything without referencing anything.