Consumer Acknowledgments and Publisher Confirms with RabbitMQ

1. Overview

In this tutorial, we’ll learn how to ensure message publication to a RabbitMQ broker with publisher confirmations. Then, we’ll see how to tell the broker we successfully consumed a message with consumer acknowledgments.

2. Scenario

In simple applications, we often overlook explicit confirmation mechanisms when using RabbitMQ, relying instead on basic message publishing to a queue and automatic message acknowledgment upon consumption. However, despite RabbitMQ’s robust infrastructure, errors can still occur, necessitating a means to double-check message delivery to the broker and confirm successful message consumption. This is where publisher confirms, and consumer acknowledgments come into play, providing a safety net.

3. Waiting for Publisher Confirms

Even without errors in our application, a published message can end up lost. For instance, it can get lost in transit due to an obscure network error. To circumvent that, AMQP provides transaction semantics to guarantee messages aren’t lost. However, this comes at a significant cost. Since transactions are heavy, the time to process messages can increase significantly, especially at large volumes.

Instead, we’ll employ the confirm mode, which, despite introducing some overhead, is faster than a transaction. This mode instructs the client and the broker to initiate a message count. Subsequently, the client verifies this count using the delivery tag sent back by the broker with the corresponding number. This process ensures the secure storage of messages for subsequent distribution to consumers.

To enter confirm mode, we need to call this once on our channel:

channel.confirmSelect();

Confirmation can take time, especially for durable queues since there’s an IO delay. So, RabbitMQ waits for confirmations asynchronously but provides synchronous methods to use in our application:

• Channel.waitForConfirms() — Blocks execution until all messages since the last call are ACK’d (acknowledged) or NACK’d (rejected) by the broker.
• Channel.waitForConfirms(timeout) — This is the same as above, but we can limit the wait to a millisecond value. Otherwise, we’ll get a TimeoutException.
• Channel.waitForConfirmsOrDie() — This one throws an exception if any message has been NACK’d since the last call. This is useful if we can’t tolerate that any messages are lost.
• Channel.waitForConfirmsOrDie(timeout) — Same as above, but with a timeout.

3.1. Publisher Setup

Let’s start with a regular class to publish messages. We’ll only receive a channel and a queue to connect to:

class UuidPublisher {
    private Channel channel;
    private String queue;
    public UuidPublisher(Channel channel, String queue) {
        this.channel = channel;
        this.queue = queue;
    }
}

Then, we’ll add a method for publishing String messages:

public void send(String message) throws IOException {
    channel.basicPublish("", queue, null, message.getBytes());
}

When we send messages this way, we risk losing them during transit, so let’s include some code to ensure that the broker safely receives our messages.

3.2. Starting Confirm Mode on a Channel

We’ll start by modifying our constructor to call confirmSelect() on the channel at the end. This is necessary so we can use the “wait” methods on our channel:

public UuidPublisher(Channel channel, String queue) throws IOException {
    // ...
    this.channel.confirmSelect();
}

If we try to wait for confirmations without entering confirm mode, we’ll get an IllegalStateException. Then, we’ll choose one of the synchronous wait() methods and call it after publishing a message using our send() method. Let’s go with waiting with a timeout so we can ensure we’ll never wait forever:

public boolean send(String message) throws Exception {
    channel.basicPublish("", queue, null, message.getBytes());
    return channel.waitForConfirms(1000);
}

Returning true means the broker successfully received the message. This works well if we’re sending a few messages.

3.3. Confirming Published Messages in Batches

Since confirming messages takes time, we shouldn’t wait for confirmation after each publication. Instead, we should send a bunch of them before waiting for confirmation. Let’s modify our method to receive a list of messages and only wait after sending all of them:

public void sendAllOrDie(List<String> messages) throws Exception {
    for (String message : messages) {
        channel.basicPublish("", queue, null, message.getBytes());
    }
    channel.waitForConfirmsOrDie(1000);
}

This time, we’re using waitForConfirmsOrDie() because a false return with waitForConfirms() would mean the broker NACK’d an unknown number of messages. While this ensures we’ll get an exception if any of the messages are NACK’d, we can’t tell which failed.

4. Leveraging Confirm Mode to Guarantee Batch Publishing

When using confirm mode, it’s also possible to register a ConfirmListener on our channel. This listener takes two callback handlers: one for successful deliveries and another for broker failures. This way, we can implement a mechanism to ensure no message is left behind. We’ll start with a method that adds this listener to our channel:

private void createConfirmListener() {
    this.channel.addConfirmListener(
      (tag, multiple) -> {
        // ...
      }, 
      (tag, multiple) -> {
        // ...
      }
    );
}

In the callbacks, the tag parameter refers to the message’s sequential delivery tag, while multiple indicates whether this confirms various messages. The tag parameter will point to the latest confirmed tag in this case. Conversely, if the last callback was a NACK, all messages with a delivery tag greater than the latest NACK callback tag are also confirmed.

To coordinate these callbacks, we’ll keep unconfirmed messages in a ConcurrentSkipListMap. We’ll put our pending messages there, using its tag number as the key. This way, we can call headMap() and get a view of all previous messages up to the tag we’re receiving now:

private ConcurrentNavigableMap<Long, PendingMessage> pendingDelivery = new ConcurrentSkipListMap<>();

The callback for confirmed messages will remove all messages up to tag from our map:

(tag, multiple) -> {
    ConcurrentNavigableMap<Long, PendingMessage> confirmed = pendingDelivery.headMap(tag, true);
    confirmed.clear();
}

The headMap() will contain a single item if multiple is false and more than one otherwise. Consequently, we don’t need to check whether we receive a confirmation for multiple messages.

4.1. Implementing a Retry Mechanism for Rejected Messages

We’ll implement a retry mechanism for the callbacks for rejected messages. Also, we’ll include a maximum number of retries to avoid a situation where we retry forever. Let’s start with a class that’ll hold the current number of tries for a message and a simple method to increment this counter:

public class PendingMessage {
    private int tries;
    private String body;
    public PendingMessage(String body) {
        this.body = body;
    }
    public int incrementTries() {
        return ++this.tries;
    }
    // standard getters
}

Now, let’s use it to implement our callback. We start by getting a view of the rejected messages, then remove any items that have exceeded the maximum number of tries:

(tag, multiple) -> {
    ConcurrentNavigableMap<Long, PendingMessage> failed = pendingDelivery.headMap(tag, true);
    failed.values().removeIf(pending -> {
        return pending.incrementTries() >= MAX_TRIES;
    });
    // ...
}

Then, if we still have pending messages, we send them again. This time, we’ll also remove the message if an unexpected error occurs in our app:

if (!pendingDelivery.isEmpty()) {
    pendingDelivery.values().removeIf(message -> {
        try {
            channel.basicPublish("", queue, null, message.getBody().getBytes());
            return false;
        } catch (IOException e) {
            return true;
        }
    });
}

4.2. Putting It All Together

Finally, we can create a new method that sends messages in a batch but can detect rejected messages and try to send them again. We have to call getNextPublishSeqNo() on our channel to find out our message tag:

public void sendOrRetry(List<String> messages) throws IOException {
    createConfirmListener();
    for (String message : messages) {
        long tag = channel.getNextPublishSeqNo();
        pendingDelivery.put(tag, new PendingMessage(message));
        channel.basicPublish("", queue, null, message.getBytes());
    }
}

We create the listener before publishing messages; otherwise, we won’t receive confirmations. This will create a cycle of receiving callbacks until we’ve successfully sent or retried all messages.

5. Sending Consumer Delivery Acknowledgments

Before we look into manual acknowledgments, let’s start with an example without them. When using automatic acknowledgments, a message is considered successfully delivered as soon as the broker fires it to a consumer. Let’s see what a simple example looks like:

public class UuidConsumer {
    private String queue;
    private Channel channel;
    // all-args constructor
    public void consume() throws IOException {
        channel.basicConsume(queue, true, (consumerTag, delivery) -> {
            // processing...
        }, cancelledTag -> {
            // logging...
        });
    }
}

Automatic acknowledgments are activated when passing true to basicConsume() via the autoAck parameter. Despite being fast and straightforward, this is unsafe because the broker discards the message before we process it. So, the safest option is to deactivate it and send a manual acknowledgment with basickAck() on the channel, guaranteeing the message is successfully processed before it exits the queue:

channel.basicConsume(queue, false, (consumerTag, delivery) -> {
    long deliveryTag = delivery.getEnvelope().getDeliveryTag();
    // processing...
    channel.basicAck(deliveryTag, false);
}, cancelledTag -> {
    // logging...
});

In its simplest form, we acknowledge each message after processing it. We use the same delivery tag we received to acknowledge the consumption. Most importantly, to signal individual acknowledgments, we must pass false to basicAck(). This can be pretty slow, so let’s see how to improve it.

5.1. Defining Basic QoS on a Channel

Usually, RabbitMQ will push messages as soon as they’re available. We’ll set essential Quality of Service settings on our channel to avoid that. So, let’s include a batchSize parameter in our constructor and pass it to basicQos() on our channel, so only this amount of messages is prefetched:

public class UuidConsumer {
    // ...
    private int batchSize;
    public UuidConsumer(Channel channel, String queue, int batchSize) throws IOException {
        // ...
        this.batchSize = batchSize;
        channel.basicQos(batchSize);
    }
}

This helps keep messages available to other consumers while we process what we can.

5.2. Defining an Acknowledgement Strategy

Instead of sending an ACK to every message we process, we can improve performance by sending one ACK every time we reach our batch size. For a more complete scenario, let’s include a simple processing method. We’ll consider the message as processed if we can parse the message as a UUID:

private boolean process(String message) {
    try {
        UUID.fromString(message);
        return true;
    } catch (IllegalArgumentException e) {
        return false;
    }
}

Now, let’s modify our consume() method with a basic skeleton for sending batch acknowledgments:

channel.basicConsume(queue, false, (consumerTag, delivery) -> {
    String message = new String(delivery.getBody(), "UTF-8");
    long deliveryTag = delivery.getEnvelope().getDeliveryTag();
    if (!process(message)) {
        // ...
    } else if (deliveryTag % batchSize == 0) {
        // ...
    } else {
        // ...
    }
}

We’ll NACK the message if we can’t process it and check if we reached the batch size to ACK pending processed messages. Otherwise, we’ll store the delivery tag of the pending ACK so it’s sent in a later iteration. We’ll store that in a class variable:

private AtomicLong pendingTag = new AtomicLong();

5.3. Rejecting Messages

We reject messages if we don’t want or can’t process them; when rejecting, we can re-queue. Re-queueing is useful, for instance, if we’re over capacity and want another consumer to take it instead of telling the broker to discard it. We have two methods for this:

• channel.basicReject(deliveryTag, requeue) — rejects a single message, with the option to re-queue or discard.
• channel.basicNack(deliveryTag, multiple, requeue) — same as above, but with the option to reject in batches. Passing true to multiple will reject every message since the last ACK up to the current delivery tag.

Since we’re rejecting messages individually, we’ll use the first option. We’ll send it and reset the variable if there’s a pending ACK. Finally, we reject the message:

if (!process(message, deliveryTag)) {
    if (pendingTag.get() != 0) {
        channel.basicAck(pendingTag.get(), true);
        pendingTag.set(0);
    }
    channel.basicReject(deliveryTag, false);
}

5.4. Acknowledging Messages In Batches

Since delivery tags are sequential, we can use the modulo operator to check if we’ve reached our batch size. If we have, we send an ACK and reset the pendingTag. This time, passing true to the “multiple” parameter is essential so the broker knows we’ve successfully processed all messages up to and including the current delivery tag:

else if (deliveryTag % batchSize == 0) {
    channel.basicAck(deliveryTag, true);
    pendingTag.set(0);
} else {
    pendingTag.set(deliveryTag);
}

Otherwise, we just set the pendingTag to check it in another iteration. Additionally, sending multiple acknowledgments for the same tag will result in a “PRECONDITION_FAILED – unknown delivery tag” error from RabbitMQ.

It’s important to note that when sending ACKs with the multiple flag, we have to consider scenarios where we’ll never reach the batch size because there are no more messages to process. One option is to keep a watcher thread that periodically checks if there are pending ACKs to send.

6. Conclusion

In this article, we’ve explored the functionalities of publisher confirms and consumer acknowledgments in RabbitMQ, which are crucial for ensuring data safety and robustness in distributed systems.

Publisher confirmations allow us to verify successful message transmission to the RabbitMQ broker, reducing the risk of message loss. Consumer acknowledgments enable controlled and resilient message processing by confirming message consumption.

Through practical code examples, we’ve seen how to implement these features effectively, providing a foundation for building reliable messaging systems.

As always, the source code is available over on GitHub.