'What is the behavior of ADF schedule triggers when a schedule starts before the previous one ends?
I am trying to contrast and find the exact behavior of ADF schedule triggers and ADF thumbing window triggers when a trigger starts before the previous pipeline triggered by the schedule ends?
For example, let’s say we have a every 5 min schedule but the pipeline takes one hour to finish. What happens to all the every-5-min triggers that happen until the pipeline is finished?
Are the behaviors of thumbing window and schedule triggers the same in this scenario?
Solution 1:[1]
When using the ADF schedule trigger (shorter recurrence time than the amount of time pipeline takes to execute), the pipeline starts executing (In progress state) every time it is triggered irrespective of the previous pipeline run status.
Using the trigger with 3-minute recurrence interval produced the following output.
The tumbling window trigger reacts in the same way as a schedule trigger. The pipeline starts execution irrespective of the status of previous run. The trigger used has a 5-minute recurrence interval.
So, in this scenario both types of triggers behave in the same way. But tumbling window triggers have a self-dependency property which is not available with Schedule triggers. If the consecutive pipeline runs depend on each other, the self-dependency property can be used. Other significant differences between these triggers, including the self-dependency property are mentioned in the following Microsoft Q&A link.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | SaideepArikontham-MT |