The Details of Broadway Callbacks

Now that we have the tickets project set up, we can start writing some code. Once a message comes from the message broker, we want to use it to create a ticket in the database for the customer and send a confirmation email to them.

Broadway callbacks

To accomplish this, we will use the Broadway behavior to configure and start the data-ingestion pipeline. There are four callbacks available for us to implement:

• handle_message/3

• prepare_messages/2

• handle_failed/2

• handle_batch/4

This section will focus on the first three callbacks in the list.

Create bookings_pipeline.ex

First, we create a new file, bookings_pipeline.ex, in the lib folder. Then, we define the BookingsPipeline module in the following way:

defmodule BookingsPipeline do
  use Broadway

end

We also add use Broadway to bring in the Broadway behavior.

Define start_link

Let’s add the start_link/1 function now. Here’s an outline of how the implementation looks:

def start_link(_args) do
  options = [
    name: BookingsPipeline,
    producer: [
      # ...
    ],
    processors: [
      # ...
    ]
  ]

  Broadway.start_link(__MODULE__, options)
end

As we can see, it’s very similar to how start_link/1 works in other places.

The list of options

The only exception is the keyword list of options. This is where all the configuration logic lives. Broadway uses this configuration to dynamically build and start all parts of the data ingestion pipeline for us. The following configuration keys are required:

• :name: Broadway uses this as a prefix to name processes.

• :producer: This contains configuration about the source of events.

• :processors: This allows us to configure the stage processes that receive the messages and do most of the work.

The :batchers setting

There are many more settings available. We’ll look at the :batchers setting. For the full list of options, check the official documentation for Broadway.start_link/2.

We start simple, and only use the minimum required configuration to get started: