How Enterprise Applications Exchange Data Using Messaging
What is Messaging?
To understand Messaging Lets Understand first what is Synchronous and Asynchronous calls .
Synchronous
A telephone call is a synchronous form of communication. I can only communicate with the other party if the other party is available at the time I place the call.
Asynchronous
Voice mail on the other hand, allows asynchronous communication. With voice mail, when the receiver does not answer, the caller can leave him a message; later the receiver (at his convenience) can listen to the messages queued in his mailbox.Voice mail enables the caller to leave a message now so that the receiver can listen to it later, which is lot easier than trying to get the caller and the receiver on the phone at the same time. Voice mail bundles (at least part of) a phone call into a message and queues it for later consumption, this is essentially how messaging works.
Messaging is a technology that enables high-speed, asynchronous, program-to-program communication with reliable delivery. Programs communicate by sending packets of data called messages to each other.
Channels, also known as queues, are logical pathways that connect the programs and convey messages. A channel behaves like a collection or array of messages, but one that is magically shared across multiple computers and can be used concurrently by multiple applications.
A sender or producer is a program that sends a message by writing the message to a channel.
A receiver or consumer is a program that receives a message by reading (and deleting) it from a channel.
What is a Messaging System?
Messaging capabilities are typically provided by a separate software system called a messaging system or message-oriented middleware (MOM). A messaging system manages messaging the way a database system manages data persistence.
The primary purpose of a database is to make sure each data record is safely persisted, and likewise the main task of a messaging system is to move messages from the sender’s computer to the receiver’s computer in a reliable fashion.
Why we need Messaging System ?
The reason a messaging system is needed to move messages from one computer to another is that computers and the networks that connect them are inherently unreliable. Just because one application is ready to send a communication does not mean that the other application is ready to receive it. Even if both applications are ready, the network may not be working, or may fail to transmit the data properly. A messaging system overcomes these limitations by repeatedly trying to transmit the message until it succeeds.
In essence, a message is transmitted in five steps:
Create — The sender creates the message and populates it with data.
Send — The sender adds the message to a channel.
Deliver — The messaging system moves the message from the sender’s computer to the receiver’s computer, making it available to the receiver.
Receive — The receiver reads the message from the channel.
Process — The receiver extracts the data from the message.
Limited Offer - Expire Soon
This diagram also shows two important messaging concepts:
Send and forget — In step 2, the sending application sends the message to the message channel. Once that send is complete, the sender can go on to other work while the messaging system transmits the message in the background. The sender can be confident that the receiver will eventually receive the message and does not have to wait until that happens.
Store and forward — In step 2, when the sending application sends the message to the message channel, the messaging system stores the message on the sender’s computer, either in memory or on disk. In step 3, the messaging system delivers the message by forwarding it from the sender’s computer to the receiver’s computer, and then stores the message once again on the receiver’s computer. This store-and-forward process may be repeated many times, as the message is moved from one computer to another, until it reaches the receiver’s computer.
Benefits of Messaging System
1. Remote Communication
Messaging systems facilitate communication between processes, especially across machines. They abstract complexities like serialization and data transport. This is essential for distributed systems where direct memory sharing isn’t possible.
Example: A service sending notifications to mobile devices through a messaging queue like RabbitMQ or Kafka.
2. Platform/Language Integration
Messaging acts as a bridge between systems built with diverse technologies. It allows seamless interaction by using universal protocols or message formats.
Example: A .NET-based billing system communicating with a Python-based analytics tool through a shared message bus like Apache Kafka.
3. Asynchronous Communication
The "send and forget" model ensures that the sender doesn't need to wait for the receiver's acknowledgment, promoting efficiency.
Example: A payment processing system where transactions are queued and processed later, allowing the user interface to remain responsive.
4. Variable Timing
Decoupling the sender’s and receiver’s timing enables each to work at its optimal pace, enhancing throughput and efficiency.
Example: A data pipeline where producers generate data continuously while consumers process it in batches.
5. Throttling
Receivers can control the flow of messages to prevent overloading, ensuring system stability.
Example: A server limiting the number of simultaneous tasks it processes from a queue to maintain performance.
6. Reliable Communication
Messaging systems ensure messages are not lost by storing and retrying delivery. This is vital for critical data.
Example: An order processing system where messages containing order details are guaranteed to be processed, even during network disruptions.
7. Disconnected Operation
Applications can queue messages for later synchronization, enabling offline operations.
Example: A mobile app queuing user inputs for upload when the device reconnects to the internet.
8. Mediation
Messaging systems act as intermediaries, simplifying the connection between multiple systems and improving scalability and fault tolerance.
Example: A message broker like AWS SQS distributing work among several microservices while handling failover and load balancing.
9. Thread Management
Callbacks and non-blocking communication free threads for other tasks, improving application efficiency.
Example: A web server using asynchronous messaging to notify users of updates without blocking threads.
Challenges of Asynchronous Messaging
1. Complex Programming Model
Asynchronous systems rely on an event-driven model, which can complicate application logic. A straightforward method call might require multiple components like request/reply channels, correlation identifiers, and error handling queues.
2. Sequence Issues
Messages can arrive out of order due to the asynchronous nature of delivery.
3. Handling Synchronous Scenarios
Some applications require immediate responses, which asynchronous systems cannot inherently guarantee.
4. Performance Overheads
Packaging data into messages and processing them adds latency. Large data transfers in particular are inefficient in messaging systems.
5. Limited Platform Support
Some messaging systems are not universally supported across platforms, limiting interoperability.
6. Vendor Lock-In
Proprietary messaging protocols and APIs can tie organizations to specific vendors.
If you find this article useful Please leave a like , subscribe and share my blog .
Follow me on LinkedIn | X