set priority for the multiple dag runs

Question

I have a dag that I want to run multiple times say 30. But airflow can parallelly execute 16 dag runs at a time. Suppose one dag run takes longer time to execute , then among the 14 dag runs waiting , I want to run 10th dag run first. Is there any way in airflow to achieve this? I know there are ways through which we can set priority weights for different dags and tasks. But I want to assign priority weights to dag runs.

In short, I am looking for a way to set priorities for the multiple dag runs of same dag ?

Thanks.

Answer 1

You can you use pools, and limit number of parallel runs within the pool to 16, in that case you make sure that you prioritize the 16 runs from same dag.