'An error occurred (403) when calling the HeadObject operation: Forbidden in airflow (2.0.0)+

Error -

*** Failed to verify remote log exists s3://airflow_test/airflow-logs/demo/task1/2022-05-13T18:20:45.561269+00:00/1.log.
An error occurred (403) when calling the HeadObject operation: Forbidden

Dockerfile -

FROM apache/airflow:2.2.3

COPY /airflow/requirements.txt /requirements.txt

RUN pip install --no-cache-dir -r /requirements.txt

RUN pip install apache-airflow[crypto,postgres,ssh,s3,log]

USER root

# Update aptitude with new repo
RUN apt-get update

# Install software 
RUN apt-get install -y git

USER airflow

Under connection UI -

Connection Id *          - aws_s3_log_storage
Connection Type *        - S3
Host                     - <My company's internal link>. (ex - https://abcd.company.com)
Extra                    - {"aws_access_key_id": "key", "aws_secret_access_key": "key", "region_name": "us-east-1"}

Under values.yaml -

config:
    logging:
        remote_logging: 'True'
        remote_base_log_folder: 's3://airflow_test/airflow-logs'
        remote_log_conn_id: 'aws_s3_log_storage'
        logging_level: 'INFO'
        fab_logging_level: 'WARN'
        encrypt_s3_logs: 'False'
        host: '<My company's internal link>. (ex - https://abcd.company.com)'
        colored_console_log: 'False'

How did I created the bucket?

Installed awscli

used the commands -

1. aws configure
    AWS Access Key ID: <access key>

    AWS Secret Access Key: <secret key>

    Default region name: us-east-1

    Default output format: 

2. aws s3 mb s3://airflow_test --endpoint-url=<My company's internal link>. (ex - https://abcd.company.com)

I am not getting a clue on how to resolve the error. I am actually very new to airflow and helm charts.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source