At Most Once, and Sometimes Not at All

DevMentors' other modular monolith gets a whole article in this corpus about how an in-process bus is assembled — an event bus in seven files walks the construction. So this part will not walk construction. GroupFlights' dispatcher is one file, and the interesting question about it is not how it is built but what it promises, and what happens to your business facts when the process stops.

The answer is at-most-once, and the “at most” is doing real work.

The intent, in the author's words

// src/Shared/GroupFlights.Shared.Plumbing/Events/EventDispatcher.cs:11-13
//NOTE: Ten dispatcher intencjonalnie nie publikuje zdarzen natychmiast,
//inaczej mogloby powodowac nieswiadome uczestniczenie w tej samej transakcji wielu modulow
internal sealed class EventDispatcher : IEventDispatcher, IHostedService, IDisposable

Translated: "This dispatcher intentionally does not publish events immediately, otherwise it could cause multiple modules to unknowingly participate in the same transaction."

That is a good instinct, clearly stated, and the design achieves it. Dispatch runs on a timer thread inside a fresh DI scope, so a Backoffice handler reacting to a Sales event gets its own BackofficeDbContext and its own SaveChanges. It cannot enlist in Sales' unit of work; a slow Backoffice write cannot hold a Sales row lock; a Backoffice failure cannot roll back a Sales commit. For a monolith whose thesis is “these modules must be able to become services later”, that is the most important thing the design gets right.

One correction on how the sentence reads, though. There are no explicit transactions anywhere in this estate. Grep for BeginTransaction or TransactionScope across all 35 projects and you get zero hits; all thirty write sites are a bare await _dbContext.SaveChangesAsync(...). So the transaction being escaped is EF Core's implicit per-SaveChanges transaction, and the property actually bought is bought by the thread hop and the fresh scope, not by any transaction awareness in the dispatcher — which contains no transaction-related code at all.

The drain loop

// EventDispatcher.cs:33-53
private async void DoWork(object? state)
{
    try
    {
        await _semaphore.WaitAsync();
        var dispatchTasks = new List<Task>();
        while (_eventsToDispatch.TryDequeue(out var @event))
        {
            dispatchTasks.Add(DispatchAsync(@event));
        }
        await Task.WhenAll(dispatchTasks);
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, ex.Message);
    }
    finally
    {
        _semaphore.Release();
    }
}

PublishAsync is _eventsToDispatch.Enqueue(@event); return Task.CompletedTask;. A Timer calls DoWork every five seconds. That is the whole mechanism, and every delivery property falls out of those two facts.

There is no outbox and no persistence of any kind. The queue is a ConcurrentQueue<IEvent> on the managed heap. Events exist only as CLR object references.

The process dying between commit and dispatch loses the events permanently. The state change is in Postgres; the announcement is gone. With a five-second period the loss window is up to five seconds of committed-but-unannounced business facts, plus whatever was in flight. Nothing detects it — no dead-letter channel, no retry, no alert, no reconciliation job, and no way afterwards to know which events were dropped. The only repair is manual, and manual requires knowing.

It is at-most-once, and the zero case is reachable three ways. TryDequeue removes the event before dispatch begins, and no path re-enqueues on failure. Crash before the tick, crash mid-dispatch, or a handler that throws — all produce zero deliveries. Never two.

Failures are swallowed twice over. The catch logs and continues, and the already-dequeued batch is gone. And await Task.WhenAll(dispatchTasks) rethrows only the first exception. Three handlers failing in one tick means one logged and two logged nowhere at all — the aggregate task observed them, so not even UnobservedTaskException fires.

Ordering is not guaranteed. Enqueue is FIFO, but the loop dequeues the entire backlog and starts every DispatchAsync before awaiting any. Two causally-ordered events published a millisecond apart can complete in either order. Handlers for one event also run concurrently. PublishMultiple compounds it: it Task.WhenAlls a set of enqueues that are already synchronous, so ordering between one command's events is not established at publish either.

Shutdown does not drain. StopAsync is return Task.CompletedTask;. It does not stop the timer, wait for the in-flight batch, or flush the queue. A graceful shutdown discards every pending event — which is a nastier property than the crash case, because graceful shutdowns happen on every deploy.

The absences reinforce each other

Four things are missing, and the order matters:

1. No message store, so nothing can be replayed.
2. No correlation or message id. IEvent is a bare marker interface with no members — no envelope, no timestamp, no headers. With a five-second tick and concurrent fan-out there is no way to tie a dispatch log line back to the request that caused it.
3. No idempotent receiver. None of the seventeen handlers is written defensively.
4. No dead-letter or invalid-message channel.

These are not four independent gaps; they are a lock. Retry requires deduplication. Deduplication requires message ids. Replay requires persistence. You cannot add reliability to this design incrementally — the first improvement you attempt requires two others first. That is worth naming because “we will add retries later” is the standard reassurance, and here it is false.

One absence is genuinely not a gap, and fairness requires saying so. There is no format indicator or schema version, and there does not need to be: events are CLR types passed by reference, never serialised for transport. Schema drift is a compile error rather than a runtime poison message. That is exactly what in-memory buys, and it is a real advantage over a broker.

The single most interesting line in the estate

// src/Shared/GroupFlights.Shared.Plumbing/Events/Extensions.cs:9-10
services.AddSingleton<IEventDispatcher, EventDispatcher>();
services.AddHostedService<EventDispatcher>();

AddHostedService<T>() expands to TryAddEnumerable(ServiceDescriptor.Singleton<IHostedService, T>()) — an implementation-type descriptor. It does not resolve, alias or forward to the IEventDispatcher registration. So the container builds two distinct EventDispatcher instances:

• Instance one is resolved for IEventDispatcher and injected into every publishing service. StartAsync is never called on it, so its _timer is null forever. It only ever enqueues.
• Instance two is resolved for IHostedService. The host calls StartAsync, so it owns the only live Timer. Nothing ever calls PublishAsync on it. It only ever drains.

They are bridged by exactly one thing:

private static readonly ConcurrentQueue<IEvent> _eventsToDispatch = new();

Every other field — _semaphore, _timer, _logger, _serviceProvider — is per-instance. Change that one field from static to instance and the entire event system silently stops delivering. Producers enqueue into a queue nobody drains; the poller drains an empty queue forever. No exception, no log line, no startup validation, no failing test. Every HTTP request still returns 201.

Whether that is a deliberate trick or a DI accident the static keyword rescued is not recoverable from the source. What is recoverable is that the identical shape appears in the fake payment gateway — AddScoped<FakePaymentProcessor>() plus AddHostedService<FakePaymentProcessor>(), bridged by one private static readonly ConcurrentQueue. It is a house idiom, not a one-off.

The durable lesson: AddSingleton<TInterface, TImpl>() and AddHostedService<TImpl>() are two registrations of one class, not one registration seen twice. If you want the hosted service and the injected service to be the same object, you have to say so — register the singleton, then AddHostedService(sp => sp.GetRequiredService<TImpl>()).

The dead branch and the reflection

Two smaller things in DispatchAsync that reward reading.

private async Task DispatchAsync<TEvent>(TEvent @event, CancellationToken cancellationToken = default)
    where TEvent : class, IEvent
{
    …
    if (typeof(IEvent).IsAssignableFrom(typeof(TEvent)))
    {
        await DispatchDynamicallyAsync(@event, cancellationToken);
        return;
    }

    using var scope = _serviceProvider.CreateScope();
    var handlers = scope.ServiceProvider.GetServices<IEventHandler<TEvent>>();
    …
}

The if is always true, because TEvent is constrained where TEvent : class, IEvent on the line above. The strongly-typed fast path below it can never execute. Every dispatch goes through reflection.

And the branch is necessary as written: DoWork calls DispatchAsync(@event) with @event statically typed IEvent, so TEvent binds to IEvent and GetServices<IEventHandler<IEvent>>() would find nothing. The reflective path does the real work; the fast path is a fossil of an attempted optimisation that was never wired to a call site that could use it. MakeGenericType plus GetMethod plus Invoke run on every event, uncached.

The log line one level up is more interesting than it looks: every event is serialised in full, pretty-printed, at Information level, unconditionally. As a teaching device it is excellent — it is a Wire Tap that makes the whole cascade watchable in a console. In anything real it is a PII leak with no level guard, no redaction and no sampling, and this estate's events carry client names and contact details.

Next, publish then save, save then publish — the same estate demonstrating the safe ordering five times and the unsafe one once, with the failure fully traceable.