Message queuing service explained

A message queueing service is a message-oriented middleware or MOM deployed in a compute cloud using software as a service model. Service subscribers access queues and or topics to exchange data using point-to-point or publish and subscribe patterns.

It's important to differentiate between event-driven and message-driven (aka queue driven) services: Event-driven services (e.g. AWS SNS) are decoupled from their consumers. Whereas queue / message driven services (e.g. AWS SQS) are coupled with their consumers. ^[1]

Message queues can be a good buffer to handle spiky workloads but they have a finite capacity. According to Gregor Hohpe, message queues require proper mechanisms (aka flow controls) to avoid filling the queue beyond its manageable capacity and to keep the system stable. ^[2]

Goals

A message queueing service aims to eliminate the traditional overhead associated with operating in-house messaging infrastructures. These operating overheads include:

Unused capacity installed to meet peak demands
Human resources that are necessary to maintain messaging infrastructure
Projects idle time waiting for resource provisioning
Need to isolate messaging resources

Besides reducing cost, a message queueing service seeks to simplify access to messaging resources and therefore facilitate integration efforts within organizations and between them.

Benefits

A message queueing service also creates new value by providing reduced costs, enhanced performance and reliability. In order to provide those benefits, a message queueing service leverages cloud computing resources such as storage, network, memory and processing capacity. By using virtually unlimited cloud computing resources, a message queueing service provides an internet scale messaging platform.

Accessibility

A message queueing service is accessible through a variety of protocols such as Java Message Service, AMQP, REST-style APIs and web services.

Usage Examples

Patient gets admitted into a hospital out of his provider's network. Producer hospital can start sending real time events about the treatment of the patient to his physician's hospital using a message queueing service platform. The cost of integration between hospitals is marginal since they do not need to configure messaging protocols, VPNs and other details.
Information processing organization that processes events from thousands of different sources, can ask its information providers to simply place messages onto queue services and reduce integration costs.
A Call Centre can carry on servicing requests for bills to be present when the billing system is unavailable
Embedded telemetry devices in vehicles can securely communicate with an application that number crunches statistics in near-real time; Round-robin messaging lets the vehicle supplier add computing resources as his sales increase.
Security trading application can post updates to P&L application that might be unavailable at the moment.
Technician submits an x-ray while consuming application instances in London, Chicago and São Paulo compete who gets the message first by listening on the same queue.

Vendors

IBM MQ^[3] IBM MQ offers a managed service that can be used on IBM Cloud and Amazon Web Services.

^[4] Service Bus offers queues, topics & subscriptions, and rules/actions in order to support publish-subscribe, temporal decoupling, and load balancing scenarios. Azure Service Bus is built on AMQP allowing any existing AMQP 1.0 client stack to interact with Service Bus directly or via existing .Net, Java, Node, and Python clients. Standard and Premium tiers allow for pay as you go or isolated resources at massive scale.

Oracle Messaging Cloud Service :^[5] This service provides a messaging solution for applications for asynchronous communication and is influenced by the Java Message Service (JMS) API specification. Any application platform that understands HTTP can also use Oracle Messaging Cloud Service through the REST interface. For Java applications, Oracle Messaging Cloud Service provides a Java library that implements and extends the JMS 1.1 interface. The Java library implements the JMS API by acting as a client of the REST API.

^[6] Supports messages natively up to 256K, or up to 2GB by transmitting payload via S3. Highly scalable, durable and resilient. Provides loose-FIFO and 'at least once' delivery in order to provide massive scale. Supports REST API and optional Java Message Service client. Low latency. Utilizes Amazon Web Services.

IronMQ

^[7] Supports messages up to 64k; guarantees order; guarantees once only delivery; no delays retrieving messages. Supports REST API and beanstalkd open source protocol. Runs on multiple clouds including AWS and Rackspace. Scaling must be managed by user.

StormMQ:^[8] Open platform supports messages up to 50Mb. Uses AMQP to avoid vendor lock-in and provide language neutrality. Locate-It Option allows customers to audit the location of their data at all times and satisfy data protection principles.

^[9] An enterprise multi-tenant, cloud messaging service that performs advanced asynchronous messaging scenarios between applications. Anypoint MQ is fully integrated with Anypoint Platform, offering role based access control, client application management, and connectors.

Notes and References

Book: Domain-Driven Design with Java - A Practitioner's Guide . 2022 . 9781800564763 . Chandrasekaran . Premanand . Krishnan . Karthik . Ford . Neal . Byars . Brandon . Buijze . Allard . Packt Publishing .
Book: Hohpe, Gregor . Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions . Addison-Wesley Professional . 978-0321200686.
Web site: IBM MQ on Cloud. IBM. 2018-11-16.
Web site: Microsoft Azure Service Bus. 2017-03-23.
Web site: Using Oracle Messaging Cloud Service. Oracle Help Center. en-us. 2018-04-20.
Web site: Amazon Simple Queue Service (API Version 2012-11-05). 2012-11-05.
Web site: IronMQ. ties distributed systems together by providing a reliable way to communicate between services and components. Highly available, persistent by design, with best-effort one-time delivery, IronMQ is the most industrial strength, cloud-native solution for modern application architecture.. iron.io. 2017-01-11. https://web.archive.org/web/20160805183719/https://www.iron.io/platform/ironmq/. 2016-08-05. dead.
Web site: StormMQ . 2010-05-03 . dead . https://archive.today/20110716150609/http://stormmq.com/enterprise-message-queuing . 2011-07-16.
Web site: AnypointMQ. 2016-08-30.