Vulnerability Metrics Are an Operations Discipline

Part 6 read the repository's history to find the files most likely to break next. This part is about the findings that arrive with a severity label already attached — and the worst pattern I know for handling them, which is seasonal. A scanner report or a penetration-test document lands once a year, a war room convenes for three weeks, everything else stops, and then eleven months of silence follow until the next report restarts the cycle. Every metric in this series so far fails by being gamed; vulnerability metrics fail differently. They fail by being episodic — treated as an audit event instead of what they actually are, which is a class of defect with its own queue. The fix is not more urgency. The fix is to run the numbers the way you run any other operational work: continuously measured, triaged by severity, owned by name, and reported on the same dashboard as everything else.

A backlog, not a bombshell

The first instrument is vulnerability density: the count of known vulnerabilities per unit of code, segmented by severity — critical, high, medium, low. The segmentation is not decoration; it is the entire triage model, because everything downstream of this number keys off the severity tier.

Collection is a solved problem and belongs in the pipeline, not the calendar. Static analysis rules in SonarQube surface issues in your own code; Snyk and Dependabot watch the dependency tree; tools like Semgrep let you encode the patterns you care about. What matters is that these run continuously — on every merge, not every fiscal year — so that density becomes a trend line rather than an annual shock.

Read the number comparatively, the same way part 6 read defect density: the module with five times the codebase's average density is telling you something, and if that module is also on last part's hotspot list — high churn, low ownership — you have found the place where risk compounds. A codebase with high density in its critical paths is carrying active risk no matter how clean the rest of it is.

My steady-state target, stated as my practice: zero critical and high findings in production code is the only defensible resting position; everything below that tier is a queue with a clock on it. Not zero findings — that target produces theatre, disputed classifications and quietly down-tuned scanners. Zero at the top two severities, with the rest managed honestly as scheduled work.

The clock starts at detection

Density counts what exists. The second instrument measures what you do about it: remediation MTTR — the mean elapsed time from a finding being detected to its fix being deployed. This is the number I weight more heavily than density, because exposure is not a function of how many findings you have; it is a function of severity multiplied by how long each one stays open. A team that surfaces plenty of findings and closes them in days is in better shape than a team whose short list has been static since spring.

Measurement is timestamp arithmetic: detection (scanner alert, advisory publication, test finding) to deployed fix, segmented by severity. The tiers I run — and these are my recommended service levels, tuned against team capacity on estates I have worked on, not an industry standard — look like this:

Adjust the numbers to your own context; I certainly did. The load-bearing idea is not any particular figure — it is that every severity tier has a clock, and every clock has an owner who can miss it. An SLA nobody is accountable for is a wish. What a short MTTR actually evidences is organisational health: clear ownership, understood priorities, and a fix path that is not blocked behind three approvals and a release train.

One measurement trap, inherited from every averaged metric in this series: the mean hides the tail. A queue that closes forty findings in three days each and leaves one open for two hundred looks healthy on the mean and is not. Track P90 or P95 alongside the mean, and treat the long tail as its own finding — an ancient open item at medium severity is usually a sign that something about it is politically, not technically, hard.

A hotspot is a question, not a verdict

The third instrument needs a distinction that SonarQube draws well, and which I have watched teams blur at real cost. In SonarQube's model, a vulnerability is a confirmed problem — the analyser is telling you the code is wrong. A security hotspot is different: it marks security-sensitive code — authentication, authorisation, cryptography, input handling, serialisation — where a problem is possible and a human needs to look, then explicitly resolve the question one way or the other: safe as written, or in need of a fix.

Blur that distinction in either direction and you lose. Treat hotspots as vulnerabilities and you drown the queue in unconfirmed noise, which is how scanners get muted. Treat them as ignorable and you accumulate the more dangerous thing: unknown risk — code in the sensitive areas that no one has ever actually looked at with the right question in mind.

Hence the third metric: hotspot review coverage, the percentage of raised hotspots that a human has reviewed and resolved. My threshold is deliberately absolute: one hundred per cent review coverage in the sensitive modules. Not because every hotspot hides a problem — most do not — but because the metric's value is in what it makes false. At full coverage, “nobody has looked at this” is no longer a true sentence anywhere in the code that handles credentials, money or user input. Review is bounded, deskable work: an engineer, the diff, a question, a recorded answer. Which is to say — it is operations.

Where it lands in the cadence

If this discipline is operational, it should be visible exactly where the rest of the operational work is visible, at the same rhythms. Here is how the three instruments slot into the cadences this series has been building:

• Per pull request: a new critical or high finding blocks the merge. This is one of only two blocking gates I run — the other being the complexity gate from part 2 — because unlike coverage, this number cannot be satisfied by a hollow gesture. The fix for a blocking finding is fixing it.
• Per sprint: the queue is walked like any other queue — new findings triaged into tiers, clocks checked, anything approaching its deadline escalated by the team lead.
• Quarterly: MTTR by severity with the P90 alongside the mean, the density trend, and hotspot review coverage — on the same quarterly health review as the debt ratio and the hotspot scores, not in a separate report with a scarier cover page.

That last clause is the thesis of this part. The moment these numbers live in their own annual document, produced for an audit and read by nobody who ships code, they stop informing anything — the queue freezes between reports and the war-room cycle begins again. On the shared dashboard, at the shared cadence, they are just another operational signal that is either trending the right way or prompting a conversation.

And the standing rule from part 1 applies with unusual force here. Judge individuals on finding counts and you will not get fewer vulnerabilities — you will get disputed severities, findings reclassified as false positives, and scanner rules quietly disabled. The instrument measures the system: the pipeline that lets findings in, and the queue discipline that gets them out.

The checklist I actually run

• Scanning runs in the pipeline, continuously — the annual report is a summary of the year's queue, never the moment of discovery.
• Density segmented by severity and read comparatively; cross-reference against the hotspot files from part 6.
• Zero critical and high findings in production as the steady state; every other tier on a clock.
• MTTR per severity tier, with P90 alongside the mean; the long tail investigated as its own finding.
• Hotspot review coverage at one hundred per cent in modules touching authentication, money or user input.
• New critical and high findings block merges; nothing else in this family does.

That closes the code half of the series: complexity, maintainability, structure, test honesty, history and now the findings queue. From here the instruments turn outward, to how fast and how reliably all of that code actually reaches production — starting with the most famous delivery metrics in the industry, which most dashboards still count incorrectly. Next, five metrics, not four.