RabbitMQ Advanced Patterns

Basic RabbitMQ is a queue and a consumer. Production RabbitMQ is replication, partition handling, cross-datacentre movement and retry topologies — the features that decide whether a node failure is a non-event or an outage.

Quorum queues#

@Bean
Queue orders() {
    return QueueBuilder.durable("q.orders")
            .quorum()
            .withArgument("x-quorum-initial-group-size", 3)
            // After 5 delivery attempts the message is dead-lettered instead of
            // looping forever. Classic queues have no equivalent.
            .deliveryLimit(5)
            .deadLetterExchange("orders.dlx")
            .build();
}

A quorum queue replicates through Raft: writes are acknowledged once a majority of replicas have them, and a leader failure triggers an automatic election. That gives predictable behaviour under failure, which mirrored classic queues never quite managed — their failure modes depended on synchronisation state in ways that were hard to reason about.

Two properties to know. A quorum queue needs an odd replica count, three or five, so a majority always exists. And it keeps messages in memory and on disk for its Raft log, so very deep queues cost more memory than a classic queue — which is exactly the case for the lazy behaviour below.

Clustering and partitions#

cluster_partition_handling = pause_minority
cluster_formation.peer_discovery_backend = k8s
cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
vm_memory_high_watermark.relative = 0.6
disk_free_limit.absolute = 5GB

vm_memory_high_watermark is worth setting explicitly. When RabbitMQ crosses it, it blocks publishers — connections simply stop accepting, which looks to an application like a hang rather than an error. Setting it to 0.6 leaves headroom and makes the block happen before the OOM killer does something less graceful.

Federation and Shovel#

Both move messages between brokers; they differ in coupling and direction.

Federation creates a link where a downstream broker subscribes to an upstream exchange or queue. It is loosely coupled, survives disconnection, and suits a hub-and-spoke topology across regions.

rabbitmqctl set_parameter federation-upstream eu-broker \
  '{"uri":"amqp://user:pass@rabbit-eu.internal","expires":3600000}'
 
rabbitmqctl set_policy --apply-to exchanges federate-orders "^orders$" \
  '{"federation-upstream-set":"all"}'

Shovel is a point-to-point move: read from a source queue, publish to a destination, acknowledge only after the destination confirms. It is the right tool for one-off migrations, draining a DLQ into another cluster, or bridging two environments.

rabbitmqctl set_parameter shovel drain-dlq '{
  "src-uri": "amqp://rabbit-old.internal", "src-queue": "q.orders.dlq",
  "dest-uri": "amqp://rabbit-new.internal", "dest-queue": "q.orders",
  "ack-mode": "on-confirm"
}'

on-confirm is the setting that makes a shovel safe: the source message is only acknowledged once the destination has confirmed it, so a failure mid-transfer redelivers rather than loses.

Priority queues#

@Bean
Queue tasks() {
    return QueueBuilder.durable("q.tasks")
            .maxPriority(10)     // keep this small; each level costs an internal sub-queue
            .build();
}
 
// Publishing with a priority
template.convertAndSend("tasks", "", payload, message -> {
    message.getMessageProperties().setPriority(9);
    return message;
});

Priority queues come with a caveat that catches almost everyone: priority only applies to messages still in the queue. With a high prefetch, the broker has already pushed a batch to consumers, and a newly arrived high-priority message waits behind them. For priority to work meaningfully you need prefetch of 1 or 2, which costs throughput.

Often the better design is two queues — one for urgent work, one for normal — with consumers weighted between them. It is simpler to reason about and does not fight the prefetch setting.

Lazy queues#

@Bean
Queue archive() {
    return QueueBuilder.durable("q.archive")
            .withArgument("x-queue-mode", "lazy")   // keep messages on disk
            .build();
}

By default RabbitMQ keeps messages in memory and pages to disk under pressure — and that paging is itself expensive, so a queue that grows fast can push the broker into a memory alarm and block publishers. Lazy mode writes to disk from the start, trading a little latency for predictable memory under deep backlogs.

Use it for queues that legitimately grow large: batch processing, replay buffers, anything where a slow consumer is normal rather than a symptom.

Delayed delivery#

The rabbitmq_delayed_message_exchange plugin adds an exchange type that holds messages before routing:

@Bean
CustomExchange delayedExchange() {
    return new CustomExchange("orders.delayed", "x-delayed-message", true, false,
            Map.of("x-delayed-type", "topic"));
}
 
public void scheduleCancellation(String orderId, Duration delay) {
    template.convertAndSend("orders.delayed", "order.unpaid.check", orderId, message -> {
        message.getMessageProperties().setHeader("x-delay", delay.toMillis());
        return message;
    });
}

This handles variable per-message delays cleanly — "cancel this order if unpaid in 15 minutes" — which the TTL-plus-dead-letter approach cannot, because a queue is FIFO and a long TTL at the head blocks shorter delays behind it.

The plugin keeps delayed messages in a node-local store, so they do not survive a node failure the way a normal persistent message does. For delays you cannot afford to lose, a database table plus a scheduled job is less elegant and more durable.

A retry topology without blocking#

q.orders ──(reject)──▶ orders.dlx ──▶ q.orders.retry.30s (TTL 30s, DLX=orders)
                                              │ TTL expires
                                              ▼
                                        back to q.orders

Each retry level is a queue with a TTL and a dead-letter exchange pointing back at the main exchange. The delay happens in the broker rather than in a blocked consumer thread, which is the key difference from in-process retry. Chain several levels — 30s, 2m, 10m — and route to a terminal DLQ after the last one.

Check the x-death header to count attempts, since it records each dead-lettering with the queue, reason and count.

What to take away#

Use quorum queues with a delivery limit for anything durable. Set pause_minority so a partition costs availability rather than consistency. Reach for Shovel to move messages and Federation to link brokers. Remember priority needs low prefetch, use lazy mode for deep queues, and build retry delays in the broker rather than in blocked consumers.