Messaging Queue-Second Draft

What is messaging queue?

Messaging queue is an intermediate component between the interacting entities, typically consumers and producers. The producer produces messages and places them in the queue, while the consumer’s task is to retrieve the messages from the queue and process them. There might be multiple producers and consumers at a time interacting with the queue. Messaging queue enables asynchronous communication between interacting entities and eliminates the speed difference between them. It also decouples the end-points making the system more flexible, available, and highly scalable. Further, it also allows interconnecting heterogeneous environments making communication easier between them.

The messaging queue approach can be utilized at several different levels. It can be used for interprocess communication within one operating system and connect various micro-services in a distributed environment.

Following is an illustration of two applications interacting via a single messaging queue.

Requirements and goals of the system

Our aim is to design a distributed messaging queue, resides on several servers, that has the following functional and non-functional requirements.

Functional Requirements

Following are the actions that a client should be able to perform.

• CreateQueue: The client can create a queue via this API call. This API call can get other parameters to control the other configuration attributes of a queue being created.

• SendMessage: This API call enables a client to send messages to a queue. In addition to other parameters, this API call takes the message with a limited size. Generally, most of the popular services like Amazon SQS and Microsoft MSMQ allow a maximum size of 256KB and 4MB, respectively. There are various approaches used to send messages larger than the specified allowed sizes. One approach is to divide the message into small chunks at the sender’s side and recombine them at the receiver side. Amazon SQS provides extended libraries through which a user can send an Amazon SQS message that contains a reference to a message payload in Amazon S3. Similarly, Microsoft MSMQ provides an MQSendLargeMessage API to send a larger message than 4MB as an XML document.

• ReceiveMessage: The client can receive a message from the queue by specifying the queue ID in the API call. After processing the message, it will call DeleteMessage to delete the processed message from the queue.

• DeleteQueue: The user can delete a queue through this API call by specifying the queue ID if there are multiple.

Non-functional requirements

Our design of the messaging queue will adhere to the following non-functional requirements.

• Scalability: The system needs to be scalable by handling increased load, queues, and messages.

• Availablity: The system should ...