The Empty Test Project Is Load-Bearing

Somebody opens the solution, notices a test project containing exactly one file — its own .csproj — and deletes it. The pull request is two lines, the reviewer approves it in thirty seconds, CI goes green, and nothing happens for four months. Then a routine change to an interface in the core ships, the build passes, the container publishes, and the gateway throws MissingMethodException on its first request in production.

Part 8 traced a package identity through the feed. This part traces the same PackageReference back into the pipeline, because the two of them together produce the most instructive structural defect in the estate — and the accident that has been covering for it.

Seventeen lines of automation

name: CI

on: [push]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v1
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 2.2.108
    - name: Build with dotnet
      run: dotnet build --configuration Release
    - name: Tests
      run: dotnet test

— .github\workflows\main.yml, entire

That is the complete automation history of this repository. It was created by 406171d “Update main.yml” on 21 September 2019 — through the GitHub web editor, judging by the message — and modified exactly once, the same day, by 3bac73e “updated CI”, which removed two blank lines and appended the Tests step. Two commits, one afternoon, fifteen months before the last commit landed.

Four lines in it decide everything.

• on: [push] with no pull_request trigger. Fork pushes do not fire the base repository's push event, so a PR from a fork runs no checks at all. Both dependabot branches and the external contributor's fix arrived that way.
• dotnet-version: 2.2.108 was correct on the day it was written and was never touched again — across 16d19b9 “.NET Core 3.0 update” and 5016fdb “Core 3.1 update”, both of which moved every project in the solution. The pipeline requests an SDK two major versions older than everything it builds. Whether the build then picked 2.2.108 or a preinstalled 3.1 on the runner depended on global.json, of which this repository has none. Either the workflow was red and ignored, or it was green because the pin was silently overridden — and in that second case the pin is worse than useless, because it declares a toolchain the build does not use.
• dotnet build --configuration Release — and this is where the article turns.
• dotnet test with no --configuration, which means Debug.

The switch that forks the build graph

Every extension .csproj carries this pair, byte-identical, six times:

<ItemGroup Condition="'$(Configuration)' == 'Debug'">
    <ProjectReference Include="..\..\src\Ntrada\Ntrada.csproj" />
</ItemGroup>

<ItemGroup Condition="'$(Configuration)' == 'Release'">
    <PackageReference Include="Ntrada" Version="0.4.*" />
</ItemGroup>

In Release, the working-tree core is not in the picture at all. CI compiles src\Ntrada in isolation, then compiles the six extensions against a downloaded, wildcard-resolved, published copy of the core from nuget.org.

Follow the consequence to its end. A commit that adds a member to IExtension, renames a property on Configuration.Route, or changes a signature on IOptionsProvider will:

1. Compile the core successfully, because the core is internally consistent.
2. Compile all six extensions successfully, because they are being checked against a frozen copy of the old core.
3. Produce a green build.
4. Be catastrophically incompatible with every extension in the same commit.

The gate does not gate the thing it exists to gate. The one class of defect that a multi-package repository has CI for — a breaking change to the shared core — is precisely the class this build configuration cannot see.

It also means two developers on one commit get two different dependency graphs. dotnet build gives you a hermetic local build against your own source. dotnet build -c Release requires network access to nuget.org and accepts whatever 0.4.* resolves to today. Neither warns.

And then the accident

dotnet test has no --configuration, so it defaults to Debug, and in Debug every ProjectReference edge reappears. dotnet test discovers the two test projects and builds each one's transitive closure:

• Ntrada.Tests.Unit → src\Ntrada. Just the core.
• Ntrada.Tests.Integration → samples\Ntrada.Samples.Api → src\Ntrada plus all six extensions.

So the extensions are compiled against the working-tree core, in Debug, on every CI run. Not because anybody arranged it. Because a test project's reference chain drags the sample application in, and the sample application references everything.

Here is that project in full. It is one file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <LangVersion>latest</LangVersion>
    <IsPackable>false</IsPackable>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Mvc" Version="2.2.0" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="3.1.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.4.0" />
    <PackageReference Include="Shouldly" Version="3.0.2" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.1" />
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\..\samples\Ntrada.Samples.Api\Ntrada.Samples.Api.csproj" />
  </ItemGroup>
</Project>

— tests\Ntrada.Tests.Integration\Ntrada.Tests.Integration.csproj

Directory listing for that folder: one entry, the file above. Zero .cs files. Zero tests. dotnet test runs it, xUnit warns that the assembly contains no tests, and the runner exits 0.

The only compilation anywhere in this pipeline that checks the core against its own extensions is a side effect of a test project with no tests in it. Delete tests\Ntrada.Tests.Integration — the single most obvious cleanup any reviewer would approve — and five of the six extensions stop being compiled against the working-tree core at all, in any configuration, ever. The build stays green. The estate loses its only real integration gate and nothing says so.

That is what “load-bearing” means here, and it is exactly the kind of thing a project-by-project read cannot find. Nothing in the .csproj above signals that it is doing structural work. Nothing in the workflow references it. The dependency exists only in the intersection of a default build configuration, a conditional ItemGroup, and a reference chain three projects long.

What that project was supposed to be

Look at those two references again, because together they are a design, not an accident. Microsoft.AspNetCore.Mvc.Testing 3.1.0 plus a project reference to the exact sample gateway you would want to host. That is: spin Ntrada.Samples.Api up in memory with WebApplicationFactory<Program>, issue real HTTP requests through an HttpClient, and assert against Ntrada.Samples.Services.Orders sitting behind it.

Which matters, because of what is not tested. src\Ntrada\Handlers\DownstreamHandler.cs is 443 lines — 18% of the core's 2,482 lines, and 1.45× the size of the next-largest file — and it has zero tests. It is the gateway. Request forwarding, headers forwarding, custom request bodies, query-string binding, header modification, transformation and HTTP retries all terminate in that file. The four classes that do have tests total 165 lines between them and carry all 31 facts in the suite.

Coverage is concentrated almost perfectly inversely to risk, and there is a fair explanation. Every tested class is a small, synchronous, pure-ish function of options plus at most one collaborator: AuthenticationManager is 30 lines and takes NtradaOptions; AuthorizationManager is 39 and takes one interface; PolicyManager is 47; ExtensionProvider is 49. Now consider DownstreamHandler: it needs an IHttpClientFactory-produced HttpClient wired through a Polly policy, a RouteConfig, a live HttpContext with a real request-body stream, and four hook interfaces. Unit-testing that in 2019, before HttpMessageHandler fakes were ergonomic, is genuinely hard.

The suite is not under-tested through carelessness. It is tested exactly as far as the easy technique reached, with the hard technique scaffolded and abandoned. The empty project is not dead weight — it is a stub that makes an intention visible, and it is the strongest evidence in the estate that the maintainer knew precisely what was missing.

Credit where it belongs, too: what does exist is good. AuthenticationManagerTests is a complete decision matrix over the three-level auth cascade — seven facts, each pairing a return-value assertion with an interaction assertion on the authentication service — written in a readable snake_case_sentence convention with the arrange block pushed into a #region at the bottom and a single-expression Act() at the top. As a demonstration of what a good unit test looks like, at a teaching organisation, it succeeds completely.

Seven days, and then nineteen commits

The suite arrived in a burst. d8ad20a “tests, samples” created the projects on 9 September 2019; 35d336b, 7b11f54, 871c6a4 and ed49ec4 all landed on the 14th; 6ed7552 on the 15th; 562852b “Handler tests base class” on the 16th. One week.

Nineteen commits follow 562852b before the project stops, and among them are 45fc7e7 “Match All forwarding for any methods”, 482ea91 “Fixed client headers”, 054b0d8 “Fixed stream content dispose”, 128c3c5 “Fixed load balanced url” and 84f0882 “Fix bug missing Content-Type from response headers”. Five bug fixes to untested code, and not one of them added a test. The suite never functioned as a regression net for the code that actually broke.

There is one more absent seam worth naming, because it is four tokens wide. ExtensionProvider scans AppDomain.CurrentDomain.GetAssemblies() with no way to inject an alternative, which is why the five tests it does have must run in the ambient assembly-load state of the test host and why its fixture has to be a nested class the scan happens to find. You cannot test “two extensions with the same name”, or “an extension whose constructor throws”, or “an extension in an assembly that has not been loaded yet” — the exact failure modes from part 3. One constructor parameter, IEnumerable<Assembly> defaulting to the AppDomain call, would have made all three testable.

The one thing this pipeline gets right

It exists. It runs on every push. It runs the tests. And it was written in September 2019, weeks after GitHub Actions opened its CI/CD capability to public beta and two months before it went generally available. For a solo-maintained open-source project of that vintage, having any automation put this repository ahead of the median, and it deserves saying before the ledger closes.

The rule to take away is narrower than “write more tests”, and more useful: before you delete the obviously useless project, find out what its reference chain is holding up. In a repository where the build configuration is a semantic switch, the dependency graph and the test graph are the same graph, and the thing keeping them honest may be the file with nothing in it.

Next, the artefact CI never builds at all.