Capturing It All in Azure DevOps

Three parts of frameworks — DORA in part 8, flow in part 9, and the human dimensions in part 10 — and not one number captured yet. That is the correct order, because capture without definitions produces confident nonsense, but a framework without capture mechanics is a poster on a wall. Most teams I work with live in Azure DevOps, and the platform occupies an awkward, under-documented middle ground: it computes some of this series' metrics natively, on precise definitions that most of its users have never read; it half-supports others; and for a few — including the most famous delivery metric in the industry — it quietly offers nothing built in at all. This part is the map of which is which, drawn from Microsoft's own documentation as I write.

Two clocks, precisely defined

Azure DevOps ships lead time and cycle time as first-class charts, and the definitions are specific enough that quoting them loosely will corrupt your numbers. Per Microsoft's documentation: lead time runs from the moment a work item is created to the moment it enters the Completed state category; cycle time runs from the first time the item enters the In Progress state category to Completed.

Map those onto part 9's vocabulary and the division of labour is clean. Lead time is the requester's clock — it includes the whole backlog wait, the time between “asked for” and “anyone touched it”, which is often the majority of the elapsed total. Cycle time is the team's clock — it starts when somebody actually picks the work up. The gap between the two numbers is itself a metric: it is the price of your queue, and watching it grow is watching Little's Law send you an invoice.

Note the phrase state category. The clocks key off the category a state is mapped to, not the state's display name — which means a customised process with a “Ready for review” state parked in the wrong category is silently measuring the wrong thing. If you customise states, auditing their category mappings is a prerequisite for trusting either chart.

The widgets themselves are better than the averages people quote from them. Each renders a scatter plot of individual items with a moving-average trend line and a band of standard deviation around it. Use all three layers: the average for the trend, the band for how honest your promises can be, and the outlying dots as a pre-written list of items worth a retrospective conversation. An average without its spread is how “we deliver in about a week” coexists with the item that took two months.

The item that came back

Buried in the same documentation page is the rule I most often find teams do not know. For a work item that is completed, later reactivated, and completed again, cycle time runs from the first time the item ever entered In Progress to the final time it entered Completed — including the time it spent sitting closed in between.

Read that twice, because the design intent is the interesting part. The naive alternative — reset the clock on reactivation, or subtract the closed interval — would make cycle time launderable: close a struggling item on Friday, reopen it Monday, and the metric forgets the struggle. Azure DevOps chose the definition that remembers. An item that bounces pays for its whole biography, closed spells included, and the scatter plot shows the bounce as one honest, ugly dot rather than two flattering ones.

The practical consequences cut both ways. On the good side, the definition is Goodhart-resistant in exactly the way part 1 demands — premature closing buys nothing, so the incentive to perform completeness weakens. On the awkward side, one boomerang item that comes back after a quarter can drag a small team's moving average visibly, so the average must be read with the scatter, not instead of it. What you must not do is the workaround I have twice caught in the wild: cloning the reactivated item into a fresh one so the dashboard stays pretty. That is theatre, it destroys the defect's paper trail, and the fifth DORA metric from part 8 exists precisely to count what it hides.

Reading the cumulative flow diagram

The cumulative flow diagram is the richest single delivery chart the platform draws, and Microsoft's guidance for it is genuinely worth reading rather than skimming. The chart stacks a coloured band per board column, counting items over time, and the three flow quantities from part 9 are all present as geometry: the slope of the completed band is throughput; the vertical thickness of a band is the WIP sitting in that state; the horizontal distance across the bands is the time work takes to traverse them. One chart, three dials — the CFD is Little's Law drawn as an area chart, and a widening band over an unchanging slope is flow time growing, visible a sprint before the cycle-time chart confirms it.

Two readings from the guidance deserve emphasis because both are counter-intuitive. First, when a band bulges, the offending step is usually not the bulging one — work piles up in a state because the succeeding step is not pulling fast enough. The instinctive response, adding capacity where the bulge is, feeds the queue instead of draining it; the fix belongs one column downstream. Second, a flat chart is not calm — it is either work genuinely stalled across the system or a team that has stopped moving cards, and those two conditions look identical from the chart alone. Instrument failure and system failure share a symptom, which is one more reason the chart informs a conversation rather than replacing one.

Where the built-ins stop

Beyond the widgets, the platform's analytics story forks, and Microsoft is unusually candid about which fork to take. For getting data into Power BI, the documentation describes Analytics views alongside OData queries — and is blunt about the first option's limits. Analytics views cover Azure Boards work-item data only, as flat lists, and Microsoft states there are no plans to update the feature; its explicit recommendation is to use OData for everything except simple Boards reports. Carry that recommendation as it stands: the vendor is telling you which of its own doors is the good one. Any metric that must join across domains — work items to pipeline runs to test results — is OData territory by construction.

Then the gap that surprises everyone the first time: there is no built-in deployment-frequency widget. The most-cited metric in software delivery, the first dial on every DORA slide, and the platform offers nothing native that charts it. If you want deployment frequency out of Azure DevOps you assemble it yourself from pipeline-run history — an OData query filtered to your production pipeline's successful runs, bucketed by week — or you install one of the Marketplace extensions that does the assembly for you. Neither is difficult; the point is that it is your build, with your definition of “deployment” and “production” baked in, and that definition belongs in writing next to the chart.

Time-in-state data — the raw material for part 9's flow efficiency — sits in the same category. The platform records state transitions but ships no per-state duration breakdown; you either derive durations yourself in OData or buy the capability, most visibly via SaaSJet's Time in State — noting that it is a paid third-party extension, which is not a criticism, merely a line for the budget conversation.

My capture stack

What I actually wire up, offered as my defaults rather than doctrine. Per team, on the team's own dashboard: the cycle-time widget and the CFD, and nothing else — two charts the team reads daily beats nine it ignores. Monthly, an OData feed into Power BI: lead- and cycle-time trends, and the flow-distribution split of completed items by type. Quarterly, the assembled DORA set: pipeline-run history joined to incident-tagged work items, with every threshold labelled by report year per part 8.

And underneath all of it, the dependency that no tooling removes: every number in this part is downstream of board hygiene. If items are not moved across the board when their reality changes, the clocks measure fiction and the CFD flatlines; if production incidents are not tagged as incidents, the change fail rate is a guess wearing a percentage sign. The cheapest analytics investment on Azure DevOps is not an extension or a query — it is the team habit of keeping the board truthful, which is a five-minute agreement that makes every chart in this series mean something.

The numbers now exist, on defensible definitions, arriving without ceremony. The remaining problem is people: three audiences who need different slices of them, presented without the red-amber-green pantomime that ruins the exercise. Next, reporting to three audiences.