Seven Test Projects, Zero Tests

The four satellite repositories this series read in its second half — Ads, Analytics, Timeline and the Blazor client — ship seven test projects between them. Every one of the seven contains exactly one file, and that file is the .csproj. Add the Stories and Users repositories from the first half and it is eleven projects; add the API gateway and the estate total is twelve. Twelve xunit scaffolds, twelve coverlet.collector references, and not one [Fact]. Part 14 measured what a reference implementation is allowed to skip in the presentation layer. This part is the honest accounting for the whole read, and the skipped layer here is the one that would have caught almost everything the previous thirteen parts found.

The scaffolds

All twelve .csproj files are identical. Not similar — identical, byte for byte apart from nothing:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.9.4" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3" />
    <PackageReference Include="coverlet.collector" Version="3.0.3" />
  </ItemGroup>
</Project>

There is no <ProjectReference> to the code under test in any of them. That is the detail that turns “no tests yet” into “no tests were ever possible”: a test file added to Trill.Services.Stories.Tests.Unit could not name SendStoryHandler, because the assembly containing it is not on the compilation's reference list. The scaffolds are not empty rooms; they are rooms with no door to the building.

The intent was real. Trill.Services.Stories.Api/Program.cs:22-23 and Trill.Services.Stories.Application/Extensions.cs:11-12 each declare:

[assembly: InternalsVisibleTo("Trill.Services.Stories.Tests.Unit")]
[assembly: InternalsVisibleTo("Trill.Services.Stories.Tests.Integration")]

Four declarations granting internal access — which matters, because every command handler in the estate is internal sealed — to two assemblies that reference nothing. Somebody thought about how the tests would reach the handlers before they thought about whether the tests would compile. Users, Ads, Analytics, Timeline and Web declare no InternalsVisibleTo at all.

And scripts/test.sh, present and identical in nine repositories, is two lines:

#!/bin/bash
dotnet test

It succeeds. It reports zero tests and exits zero. A green signal over an empty suite is worse than a red one, because a red one gets fixed. There is no CI configuration anywhere in the estate to run it — no .travis.yml, no .github/, no pipeline file — although scripts/dockerize.sh branches on $TRAVIS_BRANCH and $TRAVIS_BUILD_NUMBER, so a Travis config existed once and was removed. That same script runs docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD, putting the password in the process arguments and the build log.

What one test would have caught

The uncomfortable exercise is to walk this series' findings back and ask what each would have cost to prevent.

Seven defects, seven tests, none of them clever. Three of them are round trips — write a thing, read it back, assert it is the same thing — and round trips are the cheapest test shape there is. The estate's defect profile is not a profile of hard bugs; it is a profile of code that has never been executed twice in a row by anything that checked the answer.

Only one of those seven is a design problem you could argue about. The rest are the exact class of defect that automated testing exists to make impossible.

The other thing that was never run

Testing is one half of “does this actually work”. The other is whether the artefact builds and starts, and it does not.

Every Dockerfile in the estate is the same shape:

FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
WORKDIR /app
COPY . .
RUN dotnet publish src/Trill.Services.Stories.Api -c release -o out

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1
WORKDIR /app
COPY --from=build /app/out .
ENTRYPOINT dotnet Trill.Services.Stories.Api.dll

SDK 3.1 cannot build a net5.0 project. The net5.0 update commit retargeted every .csproj and touched no Dockerfile, and the estate has been frozen for the year since. Nothing in this estate builds as a container, and for the Blazor client it is doubly moot: a standalone WebAssembly publish produces static files and no host assembly, so dotnet Trill.Web.UI.dll would have nothing to run even if it compiled. ENTRYPOINT is in shell form throughout, so the process is not PID 1 and does not receive termination signals; COPY . . precedes the restore, so no layer caches.

scripts/start.sh exports ASPNETCORE_ENVIRONMENT=local in nine repositories, and there is no appsettings.local.json in any of them. And Trill.Services.Timeline.sln lists four projects named Trill.Services.Rules.Api, Trill.Services.Rules.Core, Trill.Services.Rules.Tests.Integration and Trill.Services.Rules.Tests.Unit — none of which exists on disk — while omitting the three projects that do, including the entire Infrastructure assembly that holds RedisStorage. The solution file cannot open. It was copy-pasted from a service that was renamed or never written, and nobody ever double-clicked it.

The tooling that would have caught any of this is absent by policy rather than by accident: no .editorconfig, no analyzer package, and not one .csproj in the six repositories sets <Nullable>, <LangVersion> or <TreatWarningsAsErrors>. Nullable reference types being off is precisely why story.Author.Id, e.Story.Visibility.From and user.AddRating(...) all compile without so much as a warning on paths where the receiver can be null.

One last number, and it is the one I find most telling: across roughly 450 source files in these six repositories there are zero TODO, FIXME or HACK comments, and zero commented-out code blocks in the services. The README.md in every repository is one word: Trill. The debt in this estate is not annotated. It is silent, which is why reading it takes fourteen articles.

What it gets right, and I mean it

A retrospective that only counts defects has misread the codebase. Several things here are better than what I see in production estates, and they are worth naming precisely.

Correlation propagation across two transports. CorrelationIdFactory is a singleton over a static AsyncLocal<CorrelationIdHolder>, using the holder-nulling trick lifted from ASP.NET Core's own HttpContextAccessor. LoggingCommandHandlerDecorator pushes the id into Serilog's LogContext for the duration of the handler, so everything the handler logs is enriched without a parameter being threaded anywhere. And AppContextFactory reconstructs the same context whether the request arrived over HTTP or AMQP. Most estates give up at the process boundary.

One error envelope, three transports, zero ceremony. exception.GetType().Name.Underscore().Replace("_exception", "") turns TooShortStoryTextException into the wire code too_short_story_text, and the same { code, reason } shape is emitted over HTTP by ExceptionToResponseMapper, over RabbitMQ by ExceptionToMessageMapper, and over gRPC. Adding a domain rule adds an exception class and a public error code in one edit. It is rename-fragile — renaming a class is a breaking API change with no compiler signal — but that is a real trade for real leverage, and it is the reason the estate has thirteen distinct, translatable domain errors rather than one ValidationException.

Timeline's hybrid fan-out, from part 11: body stored once, identifiers fanned out, rehydrated with one multi-get, the sorted-set score doing the visibility window for free. That is a design decision that takes experience, and it is in 104 lines.

The decorator stack and its marker attribute, from part 5: six open generics, an assembly scan that excludes them by attribute, and a metrics decorator that classifies failures by exception base type into domain, app and system labels. Eight lines that give you a Prometheus query separating bad input from broken dependencies, per command, with no per-handler code.

Small structural discipline in unexpected places. Trill.Services.Stories.Core.csproj has no package references at all, so the domain compiles against the BCL alone. Analytics' IDatabaseProvider is three typed collection properties that eliminate a whole class of collection-name typo. Rate's guard is if (value is < -1 or > 1), a C# 9 relational pattern in production code by an author who plainly knew the language.

And one standing hazard the estate never fixed and every reader should note as a pattern rather than a value: the Users repository commits a certificate's private key and its .pfx alongside the passphrase that unlocks it, in a settings file, in the same directory tree, while the sibling Stories repository correctly ships only the public .cer it needs for verification. The lesson is the asymmetry, not the file: a repository will happily hold a private key and its password, the .gitignore will not stop it, and the only defence is a scanner in the commit path. Rotation is the other half of that story and it was never written; the certificate expired in 2021.

The scoreboard

If I had to state the posture of this estate in one line: the architecture is finished and the engineering discipline around it was never started. Every distributed-systems concern has a component — correlation, tracing, metrics, outbox, error mapping, service discovery, secrets — and every verification concern has a placeholder. That is a coherent choice for a teaching repository, and it produces exactly the failure mode you would predict: the interesting parts are correct and the boring parts are wrong, in ways that make the interesting parts unusable.

Which is what makes the next series worth reading. The same authors rebuilt this entire product a second time, three months later, as a modular monolith — and that repository has a Trill.Tests.EndToEnd project with actual scenario tests in it, a Trill.Tests.Performance project, and a benchmark project that exists to justify a single line of DI registration. Same product, same authors, same year, opposite testing posture. Whether the rewrite also fixed the bugs this series found is the question that series opens with, and the answer is not the one you would hope for.

Next series: the same app, built twice.