'Airflow Tasks running in parallel

I need to run few Airflow tasks in parallel concurrently and if one task got completed successfully, need to call the other task. How can I do that?

Ex:
Task A must run first.If successful Task B must run.
If Task A and B are successful, need to run Task C,D,E, F and G in parallel (all starting at the same time) and tasks- C, D,E , F and G are independent of each other.
ONLY if task D is success, task  D1 must be started.
My DAG code looks like below.
task A. >> task B
task B >> [C,D,E,F,G]
D >> D1 

Thanks in advance!

airflow airflow-scheduler

Solution 1:^[1]

Your task ordering looks good, although it is sloppily written.

If your tasks are not running concurrently make sure you've set the concurrency setting to more than 1 when instantiating your DAG.

Solution 2:^[2]

Your definition is perfectly fine to what you described (other than syntax issues).

Note that parallelism depends on your executor and available resources and you can not force "start at the same time"

B >> [C,D,E,F,G]

means that C,D,E,F,G can run in parallel. This statement has no control over which will run first or even if they will run in parallel. The statement just say that they can. Should you want to define order of execution in between them then you need to change the dependency.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	scr
Solution 2	Elad Kalif

'Airflow Tasks running in parallel

Solution 1:[1]

Solution 2:[2]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]