'Airflow: Trigger DAG via UI with Parameters/Config
I see that one can trigger_dag with parameters/config key-value pairs using the airflow command line:
For Apache Airflow, How can I pass the parameters when manually trigger DAG via CLI?
In my case, I would like to centralize all operations for airflow via the airflow UI (preferably no CLI access should be granted), which has a button for triggering the dags:
How can I pass the external parameters/configs via the UI?
Solution 1:[1]
You're in luck, assuming you're on a recent version of Airflow or can upgrade. Support for triggering a DAG run with a config blob was added in Airflow 1.10.8 under AIRFLOW-5843 in #5843. It's not through the same Trigger DAG icon you've pointed to, but it's through creating a DAG Run from Browse->DAG Runs->Create.
Screenshot of the new form that supports conf copied below from the pull request that added it.
The form will also validate the JSON of the conf which may actually be a step beyond the CLI.
Solution 2:[2]
With latest versions [we use 2.2.2] of Airflow we only pass config json
When we click on the Trigger/Play button available against the DAG, we get a drop down with 2 options
- Trigger DAG
- Trigger DAG w/ config
Clicking on the second option you can pass the conf in json format and that can be used in the script and its usage can be viewed while you create the configuration JSON as below.
To access configuration in your DAG use {{ dag_run.conf }}. As core.dag_run_conf_overrides_params is set to False, so passing any configuration here won't override task params.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Daniel Huang |
| Solution 2 |