The Event Hubs Module: Maps of Hubs, Rules and Consumer Groups

An Event Hubs namespace is never one resource. The moment a real workload arrives you have the namespace, three or four hubs inside it, an authorization rule per producer and per consumer, a consumer group per downstream processor, and — if anyone whispered the word “audit” — capture shovelling every event into blob storage. A template that only provisions the namespace has answered the easy tenth of the question.

This is Part 6 of the Terraform Module Catalog, a companion series to Terraform on Azure that walks a production template library one module at a time; the shared chassis — workspace-keyed lookup maps fed by a platform JSON — is covered in Part 2 of the main series and I won't re-explain it. Part 5 looked at Cosmos DB, where the interesting config was a menu of commented-out knobs. Event Hubs is the opposite: everything is live, and everything is a map.

A namespace's children, as maps of objects

Where most templates in this library map scalars per environment — a SKU string, a capacity number — the Event Hubs stack maps collections. The hubs themselves are a map of objects inside the usual workspace-keyed map:

eventhubs_list = {
  dev = {
    orders-created = {
      name                               = "evh-orders-created-dev"
      message_retention                  = 1
      partition_count                    = 2
      capture_descr_enabled              = true
      capture_descr_encoding             = "Avro"
      capture_destination_container_name = "EventHubArchive.AzureBlockBlob"
      capture_archive_name_format        = "{Namespace}/{EventHub}/{PartitionId}/{Year}/{Month}/{Day}/{Hour}/{Minute}/{Second}"
    }
    orders-settled = {
      name              = "evh-orders-settled-dev"
      message_retention = 1
      partition_count   = 2
      capture_descr_enabled = false
    }
  }
  staging = {}
  prod    = {}
}

eventhubs = lookup(local.eventhubs_list, local.env, null)

The module for_eaches over whatever map comes out, so adding a hub to an environment is adding a key — no new resources in the root, no count arithmetic. Two details in that capture block deserve a pause, because both look like things you chose and neither is.

capture_destination_container_name = "EventHubArchive.AzureBlockBlob" reads like a blob container name. It is actually the one magic destination identifier the platform accepts for blob capture — a well-known string masquerading as a free-text field. And the capture_archive_name_format is the classic Avro layout: partition-scoped, time-bucketed folders, one Avro file per hub per partition per minute-ish window. It's tempting to “improve” it; don't, until you've met the consumer. Half the tools that read capture output (and half the humans who wrote Spark jobs against it) assume exactly this shape. The template shipping the canonical string is the template quietly saving its users a week of downstream debugging.

Authorization as boolean triplets

Event Hubs authorization rules are shared-access policies with three switches, and the template models them as exactly that — at both scopes. Namespace-wide rules:

eventhub_namespace_authorization_rule_list = {
  dev = {
    orders-sender = {
      name   = "evhns-orders-dev-send"
      listen = false
      send   = true
      manage = false
    }
    orders-reader = {
      name   = "evhns-orders-dev-listen"
      listen = true
      send   = false
      manage = false
    }
  }
  staging = {}
  prod    = {}
}

and per-hub rules, whose objects carry an extra eventhub_name to bind them to a specific hub from the first map. I like this surface a lot. The three booleans are the actual Azure permission model, undisguised; a reviewer can see at a glance that nothing in dev holds manage, and a producer identity that suddenly gains listen shows up as a one-line diff in a pull request. Consumer groups get the same treatment — a small map of { name, eventhub_name, user_metadata } objects — so every downstream processor's checkpoint identity is declared, named and reviewable rather than created ad hoc by whichever SDK connects first.

The toggles that decide whether secrets exist

The quietest good idea in this module is a row of booleans:

primary_connection_string_enabled   = false
primary_key_enabled                 = false
secondary_connection_string_enabled = false
secondary_key_enabled               = false

These don't rotate or hide credentials — they gate whether the module materialises them at all. Flip them off and no connection string is written to Key Vault, and nothing lands in Terraform state. That last part is the point: every secret a module exports lives forever in a state blob that far more identities can read than should. The best secret handling is refusing to mint the secret — especially in an estate steering everything toward managed identity anyway. Defaulting them to false and making a team consciously opt in is exactly the right polarity.

The one customized features block

Every stack in this library opens its provider with an empty features {} — except two, and this is one of them:

provider "azurerm" {
  use_msi                    = true
  skip_provider_registration = true
  features {
    key_vault {
      purge_soft_delete_on_destroy = false
    }
  }
}

The module writes connection-string secrets into the platform's shared Key Vault. Default provider behaviour on destroy is to delete and purge secrets — gone from soft-delete, unrecoverable. For a vault the stack owns, fine; for a shared platform vault, a routine terraform destroy of one team's messaging stack should never be able to permanently erase anything. One nested boolean converts “destroy my stack” from irreversible to recoverable. It's the only place in the catalog where a provider block carries an opinion, and it's the right opinion.

On the pipeline side there's little Event-Hubs-specific to report — the standard operator console from the main series — except one architectural note: the capture destination storage account is referenced by name, resolved with a data source. That account belongs to a different stack. As everywhere in this estate, the integration contract between stacks is a naming convention, not remote state.

The defect: an empty list is not an empty map

Now the honest part. Look back at eventhubs_list: in the template as I found it (on an estate I worked on), dev held a map of objects — and staging and prod held []. An empty tuple, in a slot where every consumer expects a map. The same file did it three different ways: namespace rules used {} (correct), hub rules and consumer groups used [] (not).

It's invisible in daily life because dev is where templates get exercised. Then someone runs the first staging plan and Terraform's type unifier has to find one type that accommodates both an object-of-objects and an empty tuple. Depending on version you get an inscrutable plan-time type error, or the value degrades to any and detonates inside the module with the given "for_each" argument value is unsuitable — an error pointing at module internals the stack author has never opened, three repos away from the actual typo.

The fix costs one character ([] → {}), but the lesson is worth more: in workspace-keyed maps, every environment's value must be the same type, and “empty” has a type too. Today I'd make the module declare map(object({...})) on the variable so the mismatch fails at the boundary with a message naming the input — and I'd add that where this template says staging = "TBD" inside boolean maps (it does, twice), the same trap is loaded with a string instead of a tuple. A placeholder that type-checks is a placeholder that waits.

Next in the catalog: the module where Terraform gets the factory built and then hands the wrench to the pipeline — Part 7, Data Factory.