'How does "Stateless" Saga Orchestrator handle out of order events
So I've been researching Saga orchestrator design pattern. https://chrisrichardson.net/post/sagas/2019/12/12/developing-sagas-part-4.html
Since Chris Richardson used Food To Go as an example, Let's say we have the following MSA:
Dispatch service -> Rider service
-> Stats service
-> Admin service (Monitoring)
-> Commission service
i.e. dispatch (Logistics) service makes an assignment (appropriate Rider to Order for delivery). And this DispatchCreated event is passed to 4+ MSAs and the data is stored in their local database.
And we want to have this sequence follow
Dispatch created -> Calculate Fare -> RiderReponded (Yes/No)
-> RiderDeliveryStarted -> ...(delayed, accident, or picked up at restaurant..)
-> RiderDeliveryCompleted / RiderDeliveryFailed
From another saga orchestrator conference lecture by a Distributed Systems engineer (https://www.youtube.com/watch?v=xDuwrtwYHu8) Orchestrator itself is "Stateless", has no database, and usually has no payload - (ie. it has to be simple so that it can be HA)
And the async architecture is eventually consistent.
- DispatchCreated event is consumed by these 4 services mentioned above. And DispatchCreated event also initiates Saga Orchestrator. If Saga Orchestrator sends calculateFare() command to the Commission service, How do we ensure the DispatchCreated event is already processed by the Rider service before CalculateFare() command is sent? i.e. that dispatch data is available to Rider service before calculating fares?
Do we have to do
DispatchCreated -> Init Saga Orchestrator -> ConfirmDispatchDataExist
-> Commission Service reply True to Saga -> CalculateFare()
-> ConfirmDispatchDataExist -> Rider Service reply True to Saga
-> NotifyRider() -> ...
aka. Always ask each and every service if they have the data before issuing a command?
- How does Saga Orchestrator handle duplicate/out of order events?
Saga orchestrator itself is a stateless state machine, so this does not track each orderID's state process.
State A -> State B -> State C -> ....
<----Compenstating requests...
Let's say OrderID=1 is on State C. But due to retry logic, already processed duplicate event - state transition from state A to B arrives at the saga orchestrator. Since the saga orchestrator itself is composed of multiple instances. This second orchestrator instance now sends an already completed command to a microservice.
How do we prevent that?
a) Every consuming service has to take account of these kinds of duplicate events and reply to Saga. "This command is already processed" and the second saga orchestrator terminates. All-consuming ends have to be more complex.
b) Saga orchestrator keeps track of each order's saga progression states -> this would require a database of some sort (another failing point? and does not seem to follow the guide)
- benefit Receiving services would not need to worry about out of order events.
Many thanks in advance.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|