Compensation That Cannot Fire

The reason to reach for a saga library rather than a pair of event handlers is compensation. The happy path is just handlers; the library earns its place on the unhappy path, where it replays what was done and undoes it in order. So when a codebase adds a saga framework, the interesting question is not what the saga does. It is what happens when it fails.

Part 13 read the withdrawal flow, which has no saga class and works. This part reads the one that does.

The saga

Inflow's Saga module is eight files. It references Chronicle 3.2.1, calls services.AddChronicle(), and holds one workflow: a new customer who verifies and then makes their first deposit gets ten units of bonus money.

internal sealed class NewCustomerBonusFundsSaga : Saga<NewCustomerBonusFundsSagaData>,
    ISagaStartAction<CustomerVerified>,
    ISagaAction<WalletAdded>,
    ISagaAction<DepositCompleted>,
    ISagaAction<FundsAdded>
{
    private const decimal BonusFunds = 10;
    private const string TransferName = "new_customer_bonus";
    // ...

    public async Task HandleAsync(DepositCompleted message, ISagaContext context)
    {
        if (Data.DepositCompleted)
        {
            return;
        }

        var now = _clock.CurrentDate();
        if (now > Data.VerifiedAt.AddDays(7))
        {
            await CompleteAsync();
            return;
        }

        Data.DepositCompleted = true;
        await _messageBroker.PublishAsync(new AddFunds(Data.WalletId, Data.Currency, BonusFunds,
            TransferName));
    }

    public Task CompensateAsync(DepositCompleted message, ISagaContext context)
        => Task.CompletedTask;
    // ...
}

(src\Modules\Saga\Inflow.Modules.Saga.Api\Sagas\NewCustomerBonusFundsSaga.cs, branch master.)

It is a nice shape. Four steps, a duplicate guard, a business time window, and a completion triggered by observing the effect of its own command rather than by assuming it worked — HandleAsync(FundsAdded) calls CompleteAsync() only when the transfer name matches "new_customer_bonus". That last detail is genuinely good saga hygiene and it is easy to miss.

All four CompensateAsync overrides are => Task.CompletedTask.

No door into the undo

Chronicle compensates when a saga is rejected: the saga log is replayed backwards and every handled message gets its CompensateAsync called. I have written about that machinery in detail before, in compensation is a replay in reverse, and it works — provided something rejects.

Nothing here can.

Nothing calls Reject(). No handler in the saga does; a repository-wide search finds no call site.

No rejection event ever reaches the coordinator. The single entry point is:

internal sealed class SagaEventHandler :
    IEventHandler<CustomerVerified>,
    IEventHandler<WalletAdded>,
    IEventHandler<DepositCompleted>,
    IEventHandler<FundsAdded>
{
    // each forwards to _sagaCoordinator.ProcessAsync(message, SagaContext.Empty);
}

Four subscriptions, four success events. The estate publishes DepositRejected, WithdrawalRejected and DeductFundsRejected; the Saga module subscribes to none of them. The saga is deaf to every failure the system can report.

And an exception downstream cannot reach it either. The saga publishes its AddFunds command through IMessageBroker. With useAsyncDispatcher: true — the shipped default — that hands the message to a System.Threading.Channels writer and returns:

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    await foreach (var envelope in _messageChannel.Reader.ReadAllAsync(stoppingToken))
    {
        try
        {
            _contextAccessor.Context ??= envelope.MessageContext.Context;
            await _moduleClient.PublishAsync(envelope.Message, stoppingToken);
        }
        catch (Exception exception)
        {
            _logger.LogError(exception, exception.Message);
        }
    }
}

(Shared.Infrastructure\Messaging\Dispatchers\AsyncDispatcherJob.cs.)

The publish is fire-and-forget, and the background loop catches everything and logs it. If AddFundsHandler throws — say the wallet id is empty, or the wallet's balance is over the cap from part 3 — the exception is logged by a background service and the saga, which returned long ago, never learns.

Put together: a saga over fire-and-forget messaging cannot compensate, structurally, no matter how its compensation methods are written. The four empty CompensateAsync bodies are not laziness. They are honest — there is no code path that would call them.

To be fair, this workflow is one that arguably needs no compensation. Its only side effect is crediting ten units, and the credit is the last step. There is nothing to undo. Writing the four empty overrides is what the interface requires, and leaving them empty is the correct implementation of “this saga has nothing to unwind”. The finding is not that the bodies are empty; it is that the estate demonstrates a compensating saga framework in a configuration where compensation is unreachable, and nothing says so.

The failure mode it does have

The saga has no compensation problem. It has a state problem, and its failure mode is worse than an error.

services.AddChronicle() is called with no persistence provider, so saga state and the saga log live in memory. NewCustomerBonusFundsSagaData holds VerifiedAt, WalletId, Currency and a DepositCompleted flag. Restart the process — deploy, crash, scale event, container reschedule — and every in-flight saga's data is gone.

Now read the DepositCompleted handler again with Data freshly defaulted:

• Data.DepositCompleted is false, so the duplicate guard passes.
• Data.VerifiedAt is default(DateTime), which is 0001-01-01.
• now > Data.VerifiedAt.AddDays(7) is therefore unconditionally true.
• await CompleteAsync(); return;

The saga marks itself complete and the customer never gets the bonus. No exception, no rejection, no log line saying the window was missed rather than the state was lost. The failure mode of volatile saga state here is a silently denied entitlement, recorded as a successfully completed workflow. That is the worst possible shape for a defect: it looks like success in every observable the system has.

The same defaulting bites on ordering. Data.WalletId is set by HandleAsync(WalletAdded). If DepositCompleted were processed first, Data.WalletId would be Guid.Empty, and — but for the seven-day check firing first — the saga would publish AddFunds at an empty wallet id and produce a WalletNotFoundException in a background thread. There is a duplicate guard and no prerequisite guard.

What saves it today is a property of an unrelated class. AsyncDispatcherJob is a single BackgroundService with one await foreach over one channel, so the entire estate's message flow is serialised on one consumer and processed depth-first: CustomerVerified is fully handled — including creating the deposit account, which triggers wallet creation — before the next message is read. The saga's correctness rests on an undocumented single-threading property of a background service in another assembly. Enable the outbox, whose OutboxProcessor drains with Task.WhenAll, and that property disappears.

The flag that would break it

There is a second-order problem with the ordering argument, and it is the sort of thing that only shows up when you read two subsystems against each other.

The estate ships with "outbox": { "enabled": false }. Turn it on — which is the obvious hardening step, and the one anyone reading this repository for production ideas would take first — and InMemoryMessageBroker.PublishAsync stops handing messages to the async dispatcher and hands them to the outbox instead. The outbox is durable, transactional and everything the fire-and-forget channel is not. It also has an OutboxProcessor that drains pending messages with Task.WhenAll.

Which removes the single-consumer serialisation the saga's correctness depends on. The feature flag that fixes the estate's durability problem breaks the estate's ordering guarantee, and the two live in different modules with nothing connecting them. The saga and the outbox are, in this configuration, mutually exclusive, and no comment in either says so.

That is the most useful thing the Saga module teaches, and it is entirely accidental: a workflow whose ordering rests on an emergent property of the transport is a workflow that will break the day someone improves the transport. The defence is not to document the property. It is to make the saga state its prerequisites explicitly — if (Data.WalletId == Guid.Empty) return; and let the message be redelivered — so that ordering becomes something the saga survives rather than something it assumes.

What it would take to make it real

Four changes, none large, and they are the checklist I would apply to any saga in this shape:

• Persist the state. Chronicle takes a persistence provider; without one, every deploy is a data-loss event for in-flight workflows. This is the single change that matters most.
• Distinguish “not started” from “started long ago”. DateTime VerifiedAt should be DateTime?, and a null should be an error or a wait, never a silently-expired window. A default value that happens to satisfy your business predicate is a trap that only springs in production.
• Subscribe to the rejections. DepositRejected exists and is published. A saga that only ever hears good news is not modelling a workflow, it is modelling a hope.
• Do not publish commands you cannot observe. If the saga's own action can fail, either make the failure an event the saga subscribes to, or send the command synchronously so the exception comes back.

And one governance note, because it fits the pattern this series keeps finding: SagaModule.Policies declares a policy named "saga" that appears in nobody's permission set, and SagaModule.Use has an empty body, so none of the four message types the module copies from three other modules is covered by a contract. The module that coordinates the estate is the module that validates nothing about it.

The rule of thumb: a saga is only as durable as its data and only as reactive as its subscriptions. Count the failure events it handles before you count the compensations it declares.

That is the last of the code. Next, the honest retrospective — including the two tests that assert the bugs: the tests that document the bug.