Turning a Weather API into a Stream

Somewhere at the edge of every push-based system sits a source that refuses to push. Third-party APIs are overwhelmingly request/response: you ask, they answer, and if you want to know when something changed, you ask again. The architectural question isn't whether you'll poll — you will — it's where the polling lives and who has to know about it.

FeedR's weather service — part of the open-source DevMentors sample this series walks through — gives the cleanest small answer I know: poll inside an adapter, push outside it. One class polls weatherapi.com every five seconds; everything downstream of that class, in-process and across the wire, sees only a stream of readings. The pollution stays at the boundary. This part reads that adapter closely, because it's also FeedR's showcase for two workhorse .NET patterns — typed HttpClients and Polly resilience policies — and because it contains two honest teachable flaws.

An adapter with a streaming face

The service defines the integration's inward-facing contract first:

internal interface IWeatherFeed
{
    IAsyncEnumerable<WeatherData> SubscribeAsync(string location, CancellationToken cancellationToken);
}

Note what this signature refuses to admit: that HTTP is involved at all. It's called SubscribeAsync, it returns an async stream, and the caller consumes it exactly like the price generator from part 3:

await foreach (var weather in weatherFeed.SubscribeAsync("Cracow", stoppingToken))
{
    await _streamPublisher.PublishAsync("weather", weather);
}

That's the entire hosted service, essentially — pull readings from the adapter's stream, republish each one to the Redis weather channel (part 4's seam). The poll loop lives inside the implementation:

public async IAsyncEnumerable<WeatherData> SubscribeAsync(string location,
    [EnumeratorCancellation] CancellationToken cancellationToken)
{
    while (!cancellationToken.IsCancellationRequested)
    {
        var response = await _client.GetFromJsonAsync<WeatherApiResponse>(url, cancellationToken);
        if (response is null)
        {
            await Task.Delay(TimeSpan.FromSeconds(5), cancellationToken);
            continue;
        }

        yield return new WeatherData(/* mapped fields */);
        await Task.Delay(TimeSpan.FromSeconds(5), cancellationToken);
    }
}

An infinite async iterator around a GET — poll, yield, sleep, repeat. The [EnumeratorCancellation] attribute is the piece most people miss: it binds the token passed to WithCancellation/the enumerator to the iterator's own parameter, so cancelling the consumer's await foreach actually cancels the Task.Delay and the in-flight HTTP call inside the iterator, rather than waiting politely for the next yield. Without it, “graceful shutdown” can stall for a full poll interval.

If I'm nitpicking the loop itself: a fixed five-second delay means the cycle time is five seconds plus the API's latency, and there's no jitter — a hundred instances of this service would all hit the API in loose lockstep. PeriodicTimer (new in .NET 6, the same vintage as this code) expresses “every five seconds” more accurately, and a little randomization spreads load. Details, but details that third-party rate limiters make you care about.

The typed client and one strange retry clause

The adapter never constructs an HttpClient. It's registered through a small extension in FeedR.Shared that combines the typed-client pattern with a Polly policy — quoted in full, because the policy has a clause worth staring at:

public static IServiceCollection AddHttpApiClient<TInterface, TClient>(this IServiceCollection services)
    where TInterface : class where TClient : class, TInterface
{
    services
        .AddHttpClient<TInterface, TClient>()
        .AddPolicyHandler(GetPolicy());

    return services;

    static IAsyncPolicy<HttpResponseMessage> GetPolicy()
        => HttpPolicyExtensions
            .HandleTransientHttpError()
            .OrResult(msg => msg.StatusCode == HttpStatusCode.BadRequest)
            .WaitAndRetryAsync(3, retry => TimeSpan.FromSeconds(Math.Pow(2, retry)));
}

The good parts first, because they're genuinely good:

• AddHttpClient<TInterface, TClient> gives you IHttpClientFactory management — pooled, recycled message handlers, which is the cure for both socket exhaustion (from new HttpClient() per call) and stale DNS (from one static client held forever). The adapter just declares an HttpClient constructor parameter and receives a managed one.
• The policy attaches to the registration, not the call sites. Every request through this client gets the same resilience behavior; nobody can forget to wrap a call.
• HandleTransientHttpError + exponential backoff is the correct default posture: retry 5xx, 408, and HttpRequestException at 2, 4, 8 seconds. Transient failures get absorbed; persistent ones surface after ~14 seconds.
• The generic <TInterface, TClient> shape makes this the house pattern: the next external feed gets its resilience in one registration line.

And then there's .OrResult(msg => msg.StatusCode == HttpStatusCode.BadRequest) — retry on 400. I've turned this line over for a while and I can't defend it as a general policy. A 400 means the request is malformed: same request, same answer, three retries of pure futility that add fourteen seconds of latency to every genuinely bad request. My best guesses at intent: either it's a workaround for a specific API that returns 400 when it means 429/503 (weatherapi.com returns 400 for an unrecognized location query, which is arguably transient user input), or it's demo code making retries easy to trigger on camera. Either way, the lesson generalizes: retry policies are per-dependency contracts, not universal truths. The status codes worth retrying are exactly the ones that specific API emits transiently, and a shared “one policy for all clients” helper — the very thing that makes this extension elegant — is also how a quirk built for one API silently becomes policy for all of them. In production I'd also add jitter (thundering herds are real) and a circuit breaker so a dead upstream fails fast instead of costing every caller the full retry ladder.

Translate at the boundary, aggressively

External JSON is snake_cased, nested, and shaped by someone else's information architecture. The adapter defines private records that mirror the API's shape, pins the awkward names with attributes, and immediately maps to its own flat domain record:

private record WeatherApiResponse(Location Location, Weather Current);
private record Weather(
    [property: JsonPropertyName("temp_c")] double TempC,
    double Humidity,
    Condition Condition,
    [property: JsonPropertyName("wind_kph")] double WindKph);

yield return new WeatherData($"{response.Location.Name}, {response.Location.Country}",
    response.Current.TempC, response.Current.Humidity, response.Current.WindKph,
    response.Current.Condition.Text);

The private-ness is the design decision. Nothing outside this file can ever depend on temp_c or the vendor's nesting; when the vendor reshuffles their schema (they will), the blast radius is one adapter. WeatherData(Location, Temperature, Humidity, Wind, Condition) — flat, unit-implicit, vendor-free — is what crosses the Redis channel to the aggregator. This is anti-corruption layering at its smallest useful size, and it's the discipline I most consistently see skipped when deadlines bite: the vendor DTO leaks into the domain, and two years later a weather provider migration touches forty files.

The API key, and the config lesson

Right at the top of the adapter sits this, with an apologetic TODO above it:

//TODO: Move these values to dedicated api settings and options type
private const string ApiKey = "...";   // a real key, committed to the repository
private const string ApiUrl = "https://api.weatherapi.com/v1/current.json";

The sample knows it's wrong — the TODO says so — but it shipped, and that makes it the most realistic line in the repository, because this is exactly how keys end up in git history in real teams: a const “for now”, a commit, done. Two observations worth extracting. First, the mechanical fix is cheap and the codebase already models it: the Redis options in part 4 show the IOptions<T> + configuration-section pattern; an WeatherApiOptions with key and URL, bound from configuration and fed by user-secrets locally and a vault in production, is a twenty-minute change. Second, the procedural fix is the one that matters: once a secret has touched a public commit, rotating it is the only remediation — deleting the line doesn't unpublish history. (For that reason I'm not reproducing the key here; the shape of the mistake is the lesson, not the value.)

There's also a quieter structural note in this service: the hosted service resolves IWeatherFeed from a manually created scope (_serviceProvider.CreateScope()), because typed clients are registered transient-in-scope while BackgroundServices are singletons — injecting the feed directly would capture one HttpClient (and its handler) forever, resurrecting the stale-DNS problem the factory exists to solve. It's one of those .NET lifetime rituals that looks like ceremony until you know which outage it prevents.

The pattern, portable

Strip the weather specifics and you have a template I've now used for a stock-price vendor, a shipping-status API, and an IoT gateway: an interface that promises a stream; an iterator that hides the poll; a typed client with per-dependency resilience; private DTOs and immediate translation; options-bound configuration. Downstream consumers get push semantics; the ugliness of the vendor's reality stays in one file.

Next, part 7 follows the data one hop further and one tier up: the aggregator decides some ticks are facts, and facts get an envelope, a durable broker, and an acknowledgement. Apache Pulsar enters the story.