Recalculate Only What Changed

Back in V3 I made a deliberate choice and flagged it as a debt: CreateAggregatedFacts expires everything and rebuilds everything on any change, because rebuild-it-all is simple and obviously correct. Every version since has kept that habit — change one dimension of one sale, and the fact machinery recomputes far more than the change could possibly have touched. V8 is where I finally pay down the debt. It recalculates only the measures that actually depend on what changed. This series is that intelligence: a dependency map, selective recalculation, and the honest question of whether “selective” is as selective as it claims.

The waste V8 targets

Consider what happens under the old model when you edit a sale's PaymentMethod. The SCD2 trigger fires, the hash changes, and CreateSalesFacts rebuilds the sale's measures — all of them. But PaymentMethod does not feed TotalSale, or NetRevenue, or ProfitMargin, or most of what gets rebuilt. The vast majority of that recomputation produces byte-identical values, because the inputs to those measures did not move. It is work done to confirm nothing changed.

At V3's scale that waste was cheap and I accepted it. At V7's scale — ten-million-row datasets, high change rates — recomputing eighteen measures when a change affected two is the difference between keeping up and falling behind. V8's thesis is simple: know which measures a change can affect, and recompute only those. Everything else follows from making that knowledge explicit.

The pipeline

V8 restructures recalculation into a four-stage pipeline, stated plainly in the README:

Dimension Change → Dependency Analysis → Selective Recalculation → Performance Logging

A dimension changes. The system analyses which measures depend on that dimension. It recalculates only those. And it logs what it did — how many measures it touched, how long it took — so the efficiency is measurable rather than asserted. Each stage gets its own machinery, and the rest of the series walks them: the dependency map is stage two, the selective trigger is stage three.

The foundation is a classification. Measures get a DependencyLevel from 1 to 4 — Base, Derived, Advanced, Aggregated — that captures how far a measure sits from raw inputs. A base measure depends directly on dimensions; an aggregated measure depends on other measures. That level, combined with a per-(measure, dimension) impact weight, is what lets the system reason about blast radius instead of guessing.

The numbers V8 claims

V8 is confident about its payoff, and the README puts numbers on it: “reduces recalculation overhead by up to 80%,” “60–80% reduction in recalculation time.” More usefully, it breaks the claim down per dimension, because the whole point is that different changes have different blast radii:

• Change ProductPrice → 11 of 18 measures need recalculation (a 39% reduction).
• Change Discount → 10 of 18 (44% reduction).
• Change ShippingCost → 7 of 18 (61% reduction).
• Change PaymentMethod → 0 of 18 (100% reduction).

That last line is the headline and the punchline of part 3: PaymentMethod feeds no measures at all, so editing it should recalculate nothing. Under the old rebuild-everything model it recalculated eighteen measures to produce eighteen unchanged values. Under V8 it does no measure work whatsoever. The savings are not uniform — a ProductPrice change is genuinely expensive because a lot depends on price — but they are real, and they scale with how much of the warehouse a given change actually touches.

I will hold these numbers at arm's length, as I do all the changelog claims. They are the design's intent and best case. The selective batch path turns out to be less selective than the row-level trigger, so the aggregate savings in practice depend heavily on which path runs. But the shape of the claim — work proportional to blast radius, not to schema size — is exactly right, and it is the right thing to be building toward.

What V8 inherits and adds

V8 stands on V7's batch and monitoring foundation and keeps the dependency-driven governance from V4. What it adds is the dependency-tracking apparatus:

• MeasureDimensionDependency — the map of which measures depend on which dimensions, with weights. This is the heart of part 2.
• MeasureRecalculationLog — the record of every selective recalculation: what triggered it, which measures it affected (as an array), how long it took.
• New columns everywhere: dimensions gain AffectsCalculations and CalculationPriority; measures gain DependencyLevel and CalculationComplexity; facts gain DependsOnMeasures, LastRecalculationTime, RecalculationReason.

The schema grows to know its own dependency structure. That is the real V8 change — not a new feature, but the warehouse gaining a model of itself precise enough to reason about what a change can and cannot affect.

Where the series goes

1. Recalculate only what changed — this post.
2. A dependency map between dimensions and measures — MeasureDimensionDependency, impact weights, and the priority formula.
3. The update that recalculated nothing — the selective trigger, and the PaymentMethod zero-recalc payoff.
4. Migrating a live star from V7 to V8 — dropping the old triggers, installing the new ones, and verifying the switch.
5. When selective isn't — the V8 batch path — the honest audit of where the selectivity does and doesn't hold.

After V3's blunt rebuild-everything, V8 is the smart version I promised was coming. The intelligence lives in a table, and that table is next.