One Flag, Two Failure Modes

Every messaging system has delivery semantics; most teams discover theirs during an incident. The honest ones write them down first. The DevMentors ModularMonolith estate encodes its semantics in a single configuration boolean and one ternary expression — which makes it a rare chance to read a system's reliability contract in its entirety, in about ten lines, and name what each branch actually promises. Part 7 showed what arrives broken; this part is about whether anything arrives at all.

The fork

appsettings.json carries the flag — "messaging": { "useBackgroundDispatcher": true } — and InMemoryMessageBroker.PublishAsync consults it on every publish:

public async Task PublishAsync(params IMessage[] messages)
{
    if (messages is null) return;
    messages = messages.Where(x => x is not null).ToArray();
    if (!messages.Any()) return;

    var tasks = messages.Select(x => _messagingOptions.UseBackgroundDispatcher
        ? _asynchronousDispatcher.PublishAsync(x)   // enqueue to the channel
        : _moduleClient.PublishAsync(x));           // dispatch handlers right now
    await Task.WhenAll(tasks);
}

True: the message goes into the channel and the caller resumes immediately. False: the module client runs every subscriber's handler inside the current call — inside, that is, whatever HTTP request triggered the publish. One flag, two architectures. As a design aside, the mode swap is configuration-chosen behaviour implemented as a ternary rather than as two IMessageBroker implementations selected at registration — a near-Strategy, worth contrasting with the real Strategy-by-DI the estate uses for repositories in part 10. The ternary is fine at two branches; it is the shape that grows a third branch badly.

The flag also does double duty at startup: AddMessaging registers the AsynchronousDispatcherJob hosted service only when it is true. Today both reads come from the same bound singleton, so they cannot disagree; but the decision is made in two places with nothing tying them together except that shared instance. Rebind the options per-request someday — an IOptionsSnapshot refactor is exactly the kind a well-meaning contributor makes — and the broker could enqueue into a channel no pump will ever drain.

Failure mode one: synchronous coupling

Set the flag false and trace a request. POST /conferences calls ConferenceService.AddAsync:

await _conferenceRepository.AddAsync(conference);          // row is committed
await _messageBroker.PublishAsync(new ConferenceCreated(   // handlers run HERE
    conference.Id, conference.Name, conference.ParticipantsLimit));

If Tickets' handler throws, the exception propagates up through the broker, through the service, into the middleware — and the caller gets a 500. But the first line already committed. The client is told the operation failed when the conference exists; a retry will create a duplicate. Worse, look at whose code failed: in synchronous mode, another module's handler failing turns into your module's HTTP error. Conferences' availability is now coupled to the health of every module that subscribes to it — precisely the coupling a modular monolith's compile-time walls (part 2) were built to prevent, reintroduced at runtime through the event system. The walls stop references; they do not stop exceptions.

Synchronous mode does have one real virtue: it is causally honest. When the request returns, every consequence has happened. There is no window where the conference exists but downstream state does not. Some workflows genuinely want that — and should say so by calling the other module's service explicitly, not by riding an event bus configured into synchrony.

Failure mode two: asynchronous loss

Set the flag true — the shipped default — and the coupling vanishes. Handlers run on the pump; a throwing Tickets handler is logged and dropped without touching the HTTP response. The price is a new set of windows where events cease to exist:

1. Handler throws: the pump's catch logs at Error and moves on. The event is gone; nothing will retry it.
2. Process crashes or restarts between enqueue and dispatch: the channel is memory; its contents die with the process.
3. Ordinary deployment: same as a crash, politely. Anything in flight at shutdown is lost.

That is at-most-once delivery — never stated in the code, but chosen by it, the sum of an unbounded in-memory channel, a log-and-drop catch, and no persistence anywhere.

The dual write underneath both modes

Both branches share a deeper flaw that no flag setting can fix. AddAsync performs two writes with no transaction spanning them: the row into Postgres, the event into the broker. Crash between the two awaits in async mode and the conference exists but ConferenceCreated was never published — the modules have permanently diverged, and nothing records that they did. Sync mode inverts the symptom (event handled, request failed) without removing it.

The estate's own architecture names the missing piece. The pattern is the transactional outbox: write the event into the same database transaction as the row, and let a relay publish from the outbox table afterwards. The one integration event in the system is published on the least reliable path the design could offer, in a codebase whose sibling toolkit implements the fix — Convey's outbox and inbox are the production-grade answer, from the same authors, to exactly this dual write.

Reading the ledger honestly

It is worth saying plainly what a sample owes its readers here. At-most-once is a legitimate choice — for cache invalidation, for telemetry, for anything a periodic reconciliation would heal. It is not a legitimate default assumption, and the code never surfaces the choice: no comment, no README, no interface name (IMessageBroker, not IBestEffortMessageBroker) hints that publishes are droppable. A learner absorbing this estate absorbs fire-and-forget as how event buses work. The distilled rule for your own systems: a delivery guarantee that lives only in the implementation is not a guarantee, it is a current behaviour — write the semantics where callers can see them, because the flag's two branches differ in everything except their method signature.

And there is history in this file. This same broker spent four months — commits b06fe7a to 60840b2 — as throw new NotImplementedException() while a live endpoint called it; the full story belongs to the drift ledger in part 13.

Next, the layer below the messages: the database that was split into per-module schemas before the application ever needed it — the database splits before the app does.