Your API Gateway Cannot Proxy an Image

Proxy bugs that throw are easy. Proxy bugs that produce bytes are the ones that reach production, because every layer downstream of the corruption reports success. The status code is right. The headers are right. Content-Length agrees with the body, because something recomputed it. The client library returns 200 and hands your application a byte array that is not the byte array the origin sent, and the first person to notice is a user looking at a broken image four weeks later.

Part 11 accounted for the gateway's lifetime decisions. This part is about a single method call, made twice, that decides what kind of proxy Ntrada can be.

The call

private async Task SetSuccessResponseAsync(HttpResponse response, HttpResponseMessage httpResponse,
    ExecutionData executionData)
{
    const string responseDataKey = "response.data";
    var content = await httpResponse.Content.ReadAsStringAsync();

— DownstreamHandler.cs:308-312

and its twin on the error path:

var onError = executionData.Route.OnError;
var content = await httpResponse.Content.ReadAsStringAsync();

— DownstreamHandler.cs:290-291

Every response body the gateway receives, on every route, on every status code, goes through ReadAsStringAsync. Two things happen in that call, and both matter.

The entire body is buffered into memory. ReadAsStringAsync reads to the end before returning. A 40 MB PDF from a documents service is a 40 MB allocation in the gateway's heap — and because it exceeds 85 KB, a Large Object Heap allocation, which is not compacted by default and which is being made once per concurrent request. Ten concurrent downloads is 400 MB of LOH churn in a process whose job is to move bytes.

The bytes are decoded to a UTF-16 string. ReadAsStringAsync picks an encoding from the response's Content-Type charset, falling back to UTF-8, and decodes. Then, at the far end:

await response.WriteAsync(content);

— DownstreamHandler.cs:394

HttpResponse.WriteAsync(string) re-encodes to UTF-8 by default. So the pipeline is origin bytes → decode → UTF-16 string → encode → client bytes, and that round trip is lossless only when the origin's bytes were valid text in the encoding that was guessed.

What a PNG looks like after a round trip

Take a route proxying a thumbnail service. The origin returns image/png. ReadAsStringAsync finds no charset on image/png and falls back to UTF-8. The PNG's bytes are not valid UTF-8 — a PNG signature starts 89 50 4E 47, and 0x89 is a continuation byte with no lead byte in front of it — so .NET's decoder substitutes U+FFFD, the replacement character, for every invalid sequence. Re-encoding U+FFFD to UTF-8 emits three bytes, EF BF BD.

The response the client receives is the same image with every invalid byte sequence replaced by a three-byte replacement marker. It is longer than the original, it is not a PNG, and nothing in the chain reports an error:

• The status code is copied from the origin: 200.
• The headers are copied from the origin, except two: ExcludedResponseHeaders = {"transfer-encoding", "content-length"} at DownstreamHandler.cs:24. So the origin's Content-Length — which would no longer match — is deliberately dropped, and Kestrel computes a fresh, correct one for the corrupted body.
• Content-Type is copied through, so the client is told it is receiving image/png.

The one signal that would have surfaced the corruption as a protocol error was excluded on purpose, for an unrelated and entirely correct reason. Excluding hop-by-hop headers from a proxy is textbook; it is what RFC 7230 asks for. It just also removes the only end-to-end integrity check the response had.

And the README, at line 20, lists “Static content” as a feature.

There is a smaller companion defect in the same area. Twice — at DownstreamHandler.cs:292-295 and :385-388 — the handler does this:

if (executionData.Context.Request.Method is "GET" && !response.Headers.ContainsKey(ContentTypeHeader))
{
    response.Headers[ContentTypeHeader] = ContentTypeApplicationJson;
}

Any GET response arriving without a Content-Type is labelled application/json on the way out. For a JSON gateway that is a pragmatic default. For the same gateway serving a .css file from a static-content origin that omitted the header, it is a browser refusing to apply your stylesheet.

Why it buffers: one feature, universal cost

The obvious question is why a proxy would do this at all, when Stream.CopyToAsync has been the answer since forever. The answer is at the bottom of the same method:

if (onSuccess.Data is string dataText && dataText.StartsWith(responseDataKey))
{
    var dataKey = dataText.Replace(responseDataKey, string.Empty);
    if (string.IsNullOrWhiteSpace(dataKey))
    {
        await response.WriteAsync(content);
        return;
    }

    dataKey = dataKey.Substring(1, dataKey.Length - 1);
    dynamic data = new ExpandoObject();
    JsonConvert.PopulateObject(content, data);
    var dictionary = (IDictionary<string, object>) data;
    if (!dictionary.TryGetValue(dataKey, out var dataValue))
    {
        return;
    }

    switch (dataValue)
    {
        case JObject jObject:
            await response.WriteAsync(jObject.ToString());
            return;
        case JArray jArray:
            await response.WriteAsync(jArray.ToString());
            return;
        default:
            await response.WriteAsync(dataValue.ToString());
            break;
    }
}

— DownstreamHandler.cs:406-436

This is onSuccess.data: response.data.items — a declarative response-shaping feature that lets a route unwrap an envelope from the origin and return only the interesting property. It is a legitimately useful thing for a gateway to offer, it is entirely in the spirit of “no coding whatsoever”, and it requires the whole body as a parsed object.

So the body is buffered. Not for the routes that use it — for all of them. The feature is opt-in per route; the cost is universal and unconditional. One declarative feature, used by no route in either shipped sample, imposed eager full-body buffering and a text round trip on one hundred per cent of the gateway's traffic.

That is the generalisable lesson, and it is not really about strings. It is that a feature which changes the shape of the pipeline must be paid for at the route that uses it, not at the pipeline. The version that costs nothing is a branch:

var needsBody = onSuccess?.Data is not null
                || _responseHooks.Any()
                || response.StatusCode == 204;

if (!needsBody)
{
    await httpResponse.Content.CopyToAsync(response.Body);
    return;
}

var content = await httpResponse.Content.ReadAsStringAsync();
// ... existing shaping logic

Two extra lines. Routes that shape their responses pay for buffering; routes that forward pay for nothing, keep their streaming semantics, forward binary bodies byte-exact, and hold no more than a socket buffer of any single response in memory. The information needed to make that decision — onSuccess, the hook collections, the status code — is all available at the top of the method.

The rest of the transport ledger

Buffering is the headline, but it is worth putting the gateway's other transport-fidelity decisions next to it, because two of them are good and two are not.

Good: IHttpClientFactory, used properly. A named client, CreateClient("ntrada"), with a Polly transient-error policy attached at registration. No new HttpClient() per request, no static client with a stale DNS cache. In 2019 that was still an argument people were having, and this codebase is on the right side of it.

Good: hop-by-hop headers are excluded and the request-header forwarding is three-state. forwardRequestHeaders and forwardResponseHeaders both obey the inherit-unless-overridden rule from part 3, so a noisy route can opt out of header forwarding without the whole gateway doing so.

Sharp: retries apply to every verb. AddTransientHttpErrorPolicy at NtradaExtensions.cs:95-103 is registered on the client, not per route, so a POST that returns 503 is retried — and StreamContent (DownstreamHandler.cs:222) cannot be re-read, so the retried attempt forwards an empty body. A pass-through POST that fails once arrives at the origin twice, the second time with nothing in it.

Sharp: the backoff uses the interval as the base. Math.Pow(http.Interval, retryAttempt) means interval: 1 gives constant backoff and interval: 0.5 gives decreasing backoff — half a second, then a quarter, then an eighth. The sample's interval: 2.0 is very nearly the only value for which the formula does what its name says. There is no HttpClient.Timeout override either, so the default 100 seconds stands, and there is no circuit breaker: a failing origin receives three times its normal load from a gateway that is holding connections open for a minute and a half apiece.

Resilience defaults are correctness decisions, not tuning. These four sit in the same file as the buffering, and together they describe a gateway that is safe for JSON APIs behind healthy origins and hazardous for anything else.

Next, the retrospective: nothing was ever frozen — what all thirteen findings have in common, and what Ntrada gets right.