Push, Don't Poll: FeedR at Ten Thousand Feet

Every dashboard I have ever inherited started life the same way: a timer and an HTTP GET. Poll the prices endpoint every two seconds, diff the result, repaint the grid. It works in the demo. Then someone opens forty browser tabs, the poll interval gets “tuned” to 500 ms, the upstream API starts rate-limiting you, and the data is still stale by definition — you only ever see the world as it looked at the last tick of your clock, not its.

The cure is to invert the arrow. Data should be pushed from where it changes to whoever cares, and the moment you say that sentence out loud you have signed up for a surprisingly deep stack of decisions: what carries the push between processes, what carries it to the client, what happens when a consumer is offline, and which events are throwaway ticks versus business facts you must not lose.

FeedR is the best compact codebase I know for studying those decisions in .NET. It is an open-source sample from the DevMentors team (Dariusz Pawlukiewicz and Piotr Gankiewicz), built live across a twelve-episode YouTube series — a “data aggregator for different feeds” in their words, and a deliberately simple one. I did not write it; I have read it end to end more times than I can count, run it, broken it, and borrowed from it. This series is the walkthrough I wish I'd had: subsystem by subsystem, with real source in hand and honest notes on where the sample's simplifications would bite in production.

The shape of the system

FeedR is six deployable .NET 6 services plus a shared library, and you can hold the whole topology in your head:

                        ┌──────────────┐
   clients ──────────►  │   Gateway    │  YARP reverse proxy
                        └──────┬───────┘
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
        ┌──────────┐    ┌──────────┐     ┌──────────┐
        │  Quotes  │    │ Weather  │     │   News   │   feed services
        └────┬─────┘    └────┬─────┘     └────┬─────┘
             └───────── Redis pub/sub ────────┘
                             │  (pricing / weather / news channels)
                             ▼
                       ┌────────────┐        ┌──────────┐
                       │ Aggregator │ ─────► │ Notifier │
                       └────────────┘ Pulsar └──────────┘
                                      "orders" topic

• Gateway — a YARP reverse proxy that is almost pure configuration. One public front door, five internal destinations, plus a transform that stamps a correlation ID on every request heading inward.
• Quotes — the star feed. A background service generates a random-walk price stream for four EUR currency pairs and publishes each tick to Redis. It also exposes a gRPC server-streaming endpoint so a client can subscribe to live prices directly.
• Weather — the “real integration” feed: it polls an external weather API on a five-second loop through a Polly-wrapped typed HttpClient and republishes each reading as a push.
• News — the minimal feed: an HTTP POST that turns a request body into a published event. Small, but it hosts the repository's only test project, and the test is more interesting than the endpoint.
• Aggregator — subscribes to the pricing and weather channels, applies (placeholder) business logic, and when something noteworthy happens publishes an OrderPlaced integration event to Apache Pulsar.
• Notifier — consumes orders from Pulsar and “notifies” (logs, in the sample) with the correlation ID recovered from message metadata.

Everything cross-cutting lives in FeedR.Shared: the streaming and messaging abstractions, Redis and Pulsar implementations, a System.Text.Json serializer service, correlation-ID middleware, and an HTTP resilience helper.

The idea that makes it worth reading: two tiers of events

Plenty of samples show you a message broker. FeedR quietly shows you two, and the split is the most production-relevant idea in the codebase.

Tier one: ticks. A currency pair updating from 1.13 to 1.14 is high-frequency, low-value-per-message data. If a consumer misses one, the next arrives in a second and supersedes it anyway. FeedR moves these over Redis pub/sub — fire-and-forget fan-out with no persistence and no acknowledgement. The publishing side is one line at the call site:

await _streamPublisher.PublishAsync("pricing", currencyPair);

Tier two: facts. When the Aggregator decides an order has been placed, that is not a tick. It is a business fact with downstream consequences, and losing it is not acceptable just because a subscriber was restarting. Those events go to Pulsar, a durable log-based broker, wrapped in an envelope that carries identity and correlation metadata:

var integrationEvent = new OrderPlaced(orderId, currencyPair.Symbol);
await _messagePublisher.PublishAsync("orders", integrationEvent);

Same verb, different guarantees — and different interfaces (IStreamPublisher vs IMessagePublisher), so the type system remembers the distinction even when you forget. Most real systems I have worked on eventually discovered this split the hard way: someone put ephemeral telemetry on the durable bus and melted it, or put money-shaped events on the ephemeral one and lost them. FeedR bakes the distinction into its architecture on day one, in a codebase small enough to read in an afternoon.

One tick's journey

To make the tour concrete, follow a single price update through the system:

1. An operator (or a curl) hits POST /feeds/quotes/pricing/start on the Gateway. YARP strips the prefix, stamps a correlation-id header, and forwards to the Quotes service.
2. The Quotes endpoint does not start anything itself. It writes a StartPricing marker record into a System.Threading.Channels channel and returns 200 immediately. A BackgroundService on the other end of that channel owns the generator's lifecycle.
3. The generator is an IAsyncEnumerable<CurrencyPair> — an async random walk that yields a tick per symbol per second. Each tick is JSON-serialized and published to the Redis pricing channel.
4. The Aggregator's background service, subscribed to pricing, receives the tick, logs it, and hands it to a pricing handler. Every tenth tick, the handler fabricates an order and publishes OrderPlaced to Pulsar.
5. The Notifier's consumer loop receives the Pulsar message, unwraps the envelope, and logs the notification with the correlation ID that rode along in the metadata.
6. Meanwhile, any gRPC client subscribed to the Quotes service's SubscribePricing stream is receiving the same ticks pushed over HTTP/2, no polling anywhere.

Six services, three transports (HTTP through YARP, Redis pub/sub between services, gRPC streaming and Pulsar at the edges), and not a single timer-driven GET after step one.

What the sample deliberately is not

Being fair to a teaching codebase means being clear about its scope. FeedR has no database, no authentication, no retries on the Redis hop, no health checks, and observability that consists of ILogger calls. Some of its internals have sharp edges I will point at without mercy in later parts — a gRPC bridge that busy-spins on a BlockingCollection, a “streaming” namespace that actually does pub/sub, a correlation chain with a silent gap in the middle, fire-and-forget tasks whose exceptions vanish. Those flaws are features for our purposes: each one marks the exact spot where a demo pattern and a production pattern diverge, and finding those spots is the point of reading other people's systems.

Where the series goes

1. Push, don't poll: FeedR at ten thousand feet — this post.
2. One front door for five services — the YARP gateway: routing from config, prefix-stripping transforms, per-environment destinations.
3. A channel between the endpoint and the loop — how the Quotes feed decouples HTTP from work with System.Threading.Channels.
4. The streaming seam — IStreamPublisher, no-op defaults, DI last-wins, and the honest difference between Redis pub/sub and real streams.
5. Streaming prices over gRPC — the proto contract, server streaming, fixed-point decimals, and a busy-wait bridge that deserves a rewrite.
6. Turning a weather API into a stream — polling on the inside, pushing on the outside; typed clients and a Polly policy with one strange clause.
7. Every tenth tick becomes an order — the Aggregator, MessageEnvelope<T>, and Pulsar as the durable tier.
8. One correlation ID across three transports — where the trace survives and exactly where it breaks.
9. Testing a message you can't await — WebApplicationFactory, TaskCompletionSource, and an end-to-end test that rides real Redis.
10. Six services on a laptop — pm2, Tye, and a two-file docker compose setup, plus the config layering that makes one build run anywhere.
11. What FeedR gets right, and what production would add — the retrospective.

If your real-time story is still a setInterval around a GET, start here. The rest of the series is the walk from that timer to a system where data moves because it changed — and a running inventory of what it costs.