Delivery Guarantees: Effectively-Once
Explore the concept of effectively-once delivery guarantees in event-driven microservices. Understand how producers and consumers coordinate acknowledgments and transactional scopes to minimize data loss and avoid duplicate events. This lesson covers enabling idempotence, managing transaction scopes, and controlling offsets for reliable event processing in distributed applications.
We'll cover the following...
Effectively-once delivery means that the broker will ensure that a produced event is delivered effectively once (too obvious, right?). What this means is only one Consumers can read the event, and the broker expects acknowledgment it has processed the event.
This delivery guarantee is also known as exactly once. There are many reasons why the use of the term exactly is hotly debated. For the purposes of this course, we will continue calling it effectively once and avoid those arguments.
One approach to achieving effectively-once delivery requires transactional scopes on both the producer and the broker. Because these are separate processes, there’s always a risk that one scope will successfully commit and the other will fail before completing its commit. We can keep this risk extremely low by keeping the scope committals side by side in code without any other complexities between, but this dual transaction scope synchronization simply can not guarantee 100% reliability (although it won’t be far off!).
Just like at-least-once delivery, the Producer must wait for acknowledgment from the broker that an event was received. If the broker does not acknowledge successful receipt, then the producer will repeatedly try until it does.
Any ...