Airflow:Docker-Compose:"You are running pip as root Please use user to run pip"

Question

Requirement: To run local Airflow using official docker-compose with Airflow version 2.3.0 Issue: "You are running pip as root Please use user to run pip"

Airflow version 2.3.0 OS - macOS

I was able to run fine with 2.2.4 but not with 2.3.0

Also ran the below command too before running docker-compose up airflow-init

mkdir -p ./dags ./logs ./plugins
echo -e "AIRFLOW_UID=$(id -u)" > .env

Docker-Compose curl -LfO 'https://airflow.apache.org/docs/apache-airflow/2.3.0/docker-compose.yaml'

Answer 1

FYI this bug was introduced in 2.3.0 when they put a guard in place to make sure pip is never run as root. The problem is the init container never switches to the AIRFLOW_UID user before doing its thing, but it does install the _PIP_ADDITIONAL_REQUIREMENTS because it's just common code that all the containers run....even though the init container really doesn't need the additional requirements. So, the simple fix is you can just explicitly override that env var for the init container by adding _PIP_ADDITIONAL_REQUIREMENTS: '' in your docker-compose.yaml, in the environment section for the init container.

You can see the official fix taking that exact approach here: https://github.com/apache/airflow/pull/23517/files

Answer 2

Dockerfile contains: user: "${AIRFLOW_UID:-50000}:0

when you do "echo -e "AIRFLOW_UID=$(id -u)" > .env" AIRFLOW_UID replace with your OS user. But it seems, that docker don't get AIRFLOW_UID variable in MacOS, so docker put it on AIRFLOW_UID=0. And run from root user. In 2.3.0 you cann't install pip packages by root user

From airflow docs:

For other operating systems, you will get warning that AIRFLOW_UID is not set, but you can ignore it. You can also manually create the .env file in the same folder your docker-compose.yaml is placed with this content to get rid of the warning:

AIRFLOW_UID=50000

So try to add AIRFLOW_UID=50000 to your .env.
Or delete AIRFLOW_UID from .env. Docker take AIRFLOW_UID=50000 as default variable from Dockerfile.

//excuse for bad english