Discovery and Load Balancing Before the Mesh

Before Kubernetes DNS and service meshes made “how does service A find service B?” somebody else's problem, every microservices estate had to answer it in-process. The answers came in a standard kit: a registry (Consul, Eureka, ZooKeeper), a health check to keep the registry honest, and either client-side balancing (pick an instance yourself) or a middle tier (let a load balancer pick). Convey — DevMentors' open-source .NET toolkit, whose source I have been walking through this series — ships both flavors: Convey.Discovery.Consul and Convey.LoadBalancing.Fabio. Reading them is a compact education in what the mesh eventually abstracted away, and in why teams were so eager to let it.

Registration: existence is announced, liveness is proven

AddConsul() binds the consul section:

"consul": {
  "enabled": true,
  "url": "http://localhost:8500",
  "service": "orders-service",
  "address": "docker.for.win.localhost",
  "port": "5002",
  "pingEnabled": true,
  "pingEndpoint": "ping",
  "pingInterval": 3,
  "removeAfterInterval": 3
}

and constructs a registration whose id is the service name plus the process's IServiceId GUID (from part 2) — orders-service:8f3d... — so five replicas of the same service register as five distinct instances under one name. A hosted service performs the actual registration at startup and deregisters on shutdown.

The health check is where the design earns respect:

// Convey.Discovery.Consul/Extensions.cs
var check = new ServiceCheck
{
    Interval = ParseTime(options.PingInterval),
    DeregisterCriticalServiceAfter = ParseTime(options.RemoveAfterInterval),
    Http = $"{scheme}{options.Address}:{options.Port}{pingEndpoint}"
};
registration.Checks = new[] { check };

Consul polls your /ping endpoint every pingInterval seconds, and — the crucial line — DeregisterCriticalServiceAfter evicts the instance if it stays critical past removeAfterInterval. Without that second setting, every registry I have operated accumulates ghosts: instances that crashed without running their shutdown hooks, still listed, still receiving a share of traffic as connection errors. Graceful deregistration is a courtesy; critical-state eviction is the guarantee. If you take one operational lesson from this package, it is that a registry entry must be a claim that expires, not a fact that persists.

The /ping endpoint itself is two characters of the WebApi DSL from part 5 — Get("ping", ctx => ctx.Response.WriteAsync("pong")) — which is exactly as much health check as a registry poll needs.

Resolution: client-side balancing, with a bug worth learning from

The consuming side plugs in through the layered HTTP story from part 14: setting the client type to consul swaps in ConsulServiceDiscoveryMessageHandler, a DelegatingHandler that intercepts each outbound request, asks IConsulServicesRegistry for an instance of the logical service name, and rewrites the request URI to that instance's address. Call sites still say pricing; the handler says 10.0.3.17:5003.

The registry tries to be fair about which instance:

// Services/ConsulServicesRegistry.cs
private readonly IDictionary<string, ISet<string>> _usedServices = new Dictionary<string, ISet<string>>();

public async Task<ServiceAgent> GetAsync(string name)
{
    var services = await _consulService.GetServiceAgentsAsync(name);
    ...
    else if (services.Count == _usedServices[name].Count)
    {
        _usedServices[name].Clear();   // everyone served once — start over
    }
    return GetService(services, name);
}

It remembers which instances it has already handed out and picks randomly among the unused ones until everyone has had a turn — random-without-replacement, a perfectly reasonable balancing strategy. Except for one detail that has nothing to do with distributed systems and everything to do with dependency injection: the registry is registered transient. Every resolution gets a fresh ConsulServicesRegistry, a fresh empty _usedServices, and therefore pure random selection; the carefully written rotation state dies before the second request. The code is correct, the lifetime is wrong, and no test that exercises a single instance would ever notice. I keep this example on hand for code reviews: stateful component + transient lifetime is a bug class, and it hides best in exactly this kind of infrastructure code. (Had it been singleton, the mutable Dictionary/HashSet would then need locking — the fix is a real change, not one keyword.)

Client-side discovery also has an honest architectural cost: every consumer now depends on Consul at request time, caches nothing by default here, and must handle the found-but-dead race (the handler throws ConsulServiceNotFoundException when the name resolves to nothing). Which is roughly the point where someone on the team says: can't something else do the picking?

Fabio: the balancing decision leaves the process

That something, in the HashiCorp-era stack, was Fabio — a zero-config load balancer that reads Consul's catalog and routes fabio-host/service-name to healthy instances automatically. Convey's Convey.LoadBalancing.Fabio package makes the consuming side trivial: with the client type set to fabio, outbound requests for pricing are rewritten to the Fabio URL from the fabio section, and instance selection, health filtering and retry-on-dead all happen outside your process. The message handler gets simpler than the Consul one — a URL rewrite with no registry state at all — because the hard part moved to infrastructure.

Convey wires the registration side too: AddFabio() decorates the Consul registration with the urlprefix- tags Fabio scans for, which is why the package unconditionally drags Consul's wiring in with it (AddFabio internally ensures the Consul registration and, in a move I flagged across this whole toolkit, calls BuildServiceProvider() mid-registration to read dependencies). Fabio without Consul is not a configuration this package contemplates.

Client-side versus middle-tier is a genuine trade, and the packages embody it neatly: Consul-style gives you no extra hop and per-client policy at the cost of coupling every consumer to the registry and to correct balancing code (see above — that cost is real); Fabio-style gives you one dumb URL and centralized routing at the cost of an extra hop and a new piece of infrastructure to keep alive. In 2026 the same trade still exists — it has just been renamed to “headless service + client LB” versus “mesh sidecar.”

What the mesh made of all this

Run these services on Kubernetes today and the entire apparatus dissolves: the Deployment is the registration, readiness probes are the ping check (with eviction built in), the ClusterIP service is Fabio, and Microsoft.Extensions.ServiceDiscovery or Aspire's service bindings cover the local-dev gap. That is not a criticism of Convey — it is the strongest possible validation. The platform absorbed this layer because every fleet needed it and none of it was domain logic; the packages were load-bearing right up until the infrastructure underneath them learned the same tricks.

What survives is the checklist this code teaches: instances must expire, health must be proven not assumed, logical names belong in code and addresses in infrastructure, and any stateful component in the resolution path needs its lifetime chosen on purpose. Every one of those rules still applies inside the mesh — the mesh just enforces them in YAML instead of C#.

Next part: configuration that arrives before the app does — Convey's Vault integration, dynamic credentials and PKI.