Fanout Architecture Using Amazon SNS and SQS

Fanout is a common messaging pattern used in distributed architectures, such as event-driven architectures, where messages from one producer are sent to multiple consumers for synchronous processing. In the fanout pattern, we have the following components:

• Producers: Producers are the entities that send messages or events that need to be processed.

• Exchange point: The messages from a producer are sent to an exchange point, which is responsible for delivering the messages to their respective destinations.

• Queues: Messages sent from an exchange point are stored in a queue.

• Consumers: Consumers fetch messages from a queue and process them.

Fanout using SNS and SQS

Amazon SNS and Amazon SQS can be used to create a fanout pattern in their cloud applications. Amazon SNS uses a push mechanism to deliver time-critical messages to its subscribers, while Amazon SQS uses a polling method to allow users to fetch messages from a queue whenever they are available.

Amazon SNS can be used as an exchange point, and Amazon SQS can be used to store messages to build a fanout messaging pattern. Any messages Amazon SNS receives from a consumer will immediately be sent to its subscribers. We can add multiple SQS queues as subscribers to our SNS topic, which will store these messages and deliver them whenever a consumer requests polling.

Use case: Processing objects uploaded in a bucket

Let’s assume we want to receive an email notification whenever an object is uploaded to an S3 bucket. Also, if the object being uploaded is an image, we want to add a watermark to it and extract any text that might be available in the image. These results will then be uploaded in another bucket.

We can build this application using Amazon SNS and SQS.