'No module named 'airflow.providers.ssh' on AWS Airflow (Amazon MWAA)
I need to use sshoperator in a DAG on AWS Airflow (Amazon MWAA), so I imported the following library in my DAG file
from airflow.contrib.operators.ssh_operator import SSHOperator
It seems sshoperator has been defined in paramiko library, so I have added the following modules to requiremets.txt file as well
- paramiko>=2.6.0
- pysftp>=0.2.9
- sshtunnel>=0.1.4
But still, I'm getting the following error when I browse DAGs
Broken DAG:
File "/usr/local/lib/python3.7/site-packages/airflow/contrib/operators/ssh_operator.py",
line from airflow.providers.ssh.operators.ssh import SSHOperator
ModuleNotFoundError: No module named 'airflow.providers.ssh'
My underestanding was sshoperator in an essential and built in operators in Airflow, so technically I shouldn't add any extra libraries or plugins to Airflow.
Solution 1:[1]
This command fixed it for me:
pip install apache-airflow-providers-ssh
Solution 2:[2]
Was researching something similar and believe that I found the answer to both of our questions... According to Docker Questions...
Providers package is no longer included with Airflow, but you can separately install them with pip using the specific backport package
It looks like there is a backport for ssh here. So probably just have to add that to your requirements.txt?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | Aramis NSR |
Solution 2 | ET Come Back |