RabbitMQ Architecture & Core Concepts

RabbitMQ's power is in routing. Where Kafka gives you a partitioned log and leaves distribution to the consumer, RabbitMQ lets the broker decide which queues a message reaches based on rules you declare — which is why it stays the better fit for task distribution and complex workflows.

Exchanges and bindings#

Topic patterns are worth internalising because they cover almost every case. With a key of order.eu.created: order.*.created matches, order.eu.# matches, order.# matches, *.eu.created matches, and order.* does not — * is exactly one word, and there are three.

Design the routing key as a hierarchy from general to specific: <entity>.<region>.<event>. Consumers then subscribe at whatever granularity they need without the producer knowing they exist, which is the whole point of using an exchange.

Queue properties#

@Configuration
public class RabbitTopology {
 
    @Bean
    TopicExchange ordersExchange() {
        return ExchangeBuilder.topicExchange("orders").durable(true).build();
    }
 
    @Bean
    Queue orderCreatedQueue() {
        return QueueBuilder.durable("q.order.created")
                // Quorum queues use Raft. Classic mirrored queues are deprecated.
                .quorum()
                // Rejected or expired messages go here rather than vanishing.
                .deadLetterExchange("orders.dlx")
                .deadLetterRoutingKey("order.created.failed")
                // Redelivery limit: after this many attempts the message is
                // dead-lettered instead of looping forever.
                .deliveryLimit(5)
                .ttl(3_600_000)
                .maxLength(100_000)
                .build();
    }
 
    @Bean
    Binding orderCreatedBinding(Queue orderCreatedQueue, TopicExchange ordersExchange) {
        return BindingBuilder.bind(orderCreatedQueue).to(ordersExchange).with("order.*.created");
    }
 
    @Bean
    Queue deadLetterQueue() {
        return QueueBuilder.durable("q.order.dlq").quorum().build();
    }
}

deliveryLimit on a quorum queue is the fix for the classic poison-message loop. Without it, a message that always fails is requeued forever, consuming capacity and blocking the messages behind it.

The three parts of durability#

This trips up nearly everyone, because each part looks like it should be sufficient on its own.

A durable queue survives a broker restart as a definition — but an empty one, unless the messages were also persistent.

A persistent message (delivery_mode=2) is written to disk — but only if it is in a durable queue. A persistent message in a transient queue dies with the queue.

A durable exchange survives restart. If it does not, publishes after the restart fail because the exchange no longer exists.

Spring AMQP defaults to persistent messages, and QueueBuilder.durable(...) handles the queue. The exchange is the one people forget.

Note the cost: persistence means an fsync per message. Where throughput matters more than durability — transient notifications, cache invalidation — non-persistent messages are dramatically faster, and that is a legitimate choice as long as it is deliberate.

Prefetch and fair dispatch#

Prefetch caps how many unacknowledged messages a consumer may hold. With no limit, the broker pushes messages as fast as the socket allows, one consumer takes the lot, and scaling out has no effect — a genuinely confusing failure because everything looks correctly configured.

spring:
  rabbitmq:
    listener:
      simple:
        prefetch: 10
        concurrency: 3
        max-concurrency: 10
        acknowledge-mode: manual
        default-requeue-rejected: false

Choose the value from the work profile. Slow, variable tasks — image processing, report generation — want prefetch 1, so a consumer takes another message only when genuinely free. Fast uniform messages want 10 to 100, because the acknowledgement round trip otherwise dominates.

default-requeue-rejected: false is important: with true, a rejected message goes straight back to the head of the queue and is immediately redelivered, producing a hot loop.

Publisher confirms#

Publishing to RabbitMQ is fire-and-forget by default. The broker may reject the message, or there may be no queue bound to match the routing key, and the producer never finds out.

spring:
  rabbitmq:
    publisher-confirm-type: correlated
    publisher-returns: true
    template:
      mandatory: true

@Component
public class OrderPublisher {
 
    private final RabbitTemplate template;
 
    public OrderPublisher(RabbitTemplate template) {
        this.template = template;
 
        // The broker accepted (or rejected) the message.
        template.setConfirmCallback((correlation, ack, reason) -> {
            if (!ack) log.error("broker rejected {}: {}", correlation, reason);
        });
 
        // Accepted by the exchange but matched no queue — a routing bug that
        // is otherwise completely invisible.
        template.setReturnsCallback(returned ->
            log.error("unroutable: exchange={} key={}",
                    returned.getExchange(), returned.getRoutedKey()));
    }
}

The returns callback catches a specific and nasty class of bug. A message published with a routing key nothing is bound to is silently discarded — the publish "succeeds", and the message simply disappears. With mandatory: true plus a returns callback, you get a log line instead of a mystery.

Connections and channels#

A connection is a TCP socket and is expensive; a channel is a lightweight virtual connection multiplexed over it. The rule is one connection per application, one channel per thread — channels are not thread-safe, and sharing one across threads produces protocol errors that look like broker faults.

CachingConnectionFactory handles this for you. Give it a list of addresses so it can fail over between cluster nodes, and set a heartbeat below any proxy or firewall idle timeout on the path.

What to take away#

Publish to exchanges, let bindings route, and design routing keys as a hierarchy. Get all three parts of durability right. Set a prefetch that matches your work profile, or scaling out will not work. Turn on publisher confirms and returns — silent message loss is otherwise invisible until someone notices missing data.