Adjust Batch Size and Timeout
Learn to adjust the batch size and batch timeout to set a threshold for the maximum number of messages received and the timeout period to receive messages, respectively.
Use batch size
If we inspect batch_info.size
in the handle_batch/4
callbacks, we notice that our batches contain no more than fifty messages. The maximum number of messages received is also known as batch size and is typically controlled by the :batch_size
value for each batcher in start_link/1
. We can see this here:
...
batchers: [
...
default: [
batch_size: 100
]
]
However, the :batch_size
default value for each batcher is already 100
. It may seem like something is going wrong since we only get half the expected amount.
This is actually by design and is specific to the broadway_rabbitmq
producer. The BroadwayRabbitMQ.Producer
module maintains an active connection to the RabbitMQ server, which allows it to receive messages as soon as they get in the queue. This is why it limits the number of messages coming from RabbitMQ to control the flow and handle back-pressure. This is documented in the official broadway_rabbitmq
documentation page.
The good news is that we can easily increase this limit by passing some extra configuration to our producer_config
:
Get hands-on with 1400+ tech skills courses.