'How to use AWS CodeArtifact *within* A Dockerfile in AWSCodeBuild

I am trying to do a pip install from codeartifact from within a dockerbuild in aws codebuild.

This article does not quite solve my problem: https://docs.aws.amazon.com/codeartifact/latest/ug/using-python-packages-in-codebuild.html

The login to AWS CodeArtifct is in the prebuild; outside of the Docker context.

But my pip install is inside my Dockerfile (we pull from a private pypi registry).

How do I do this, without doing something horrible like setting an env variable to the password derived from reading ~/.config/pip.conf/ after running the login command in prebuild?



Solution 1:[1]

You can use the environment variable: PIP_INDEX_URL[1].

Below is an AWS CodeBuild buildspec.yml file where we construct the PIP_INDEX_URL for CodeArtifact by using this example from the AWS documentation.

buildspec.yml

  pre_build:
    commands:
      - echo Getting CodeArtifact authorization...
      - export CODEARTIFACT_AUTH_TOKEN=$(aws codeartifact get-authorization-token --domain "${CODEARTIFACT_DOMAIN}" --domain-owner "${AWS_ACCOUNT_ID}" --query authorizationToken --output text)
      - export PIP_INDEX_URL="https://aws:${CODEARTIFACT_AUTH_TOKEN}@${CODEARTIFACT_DOMAIN}-${AWS_ACCOUNT_ID}.d.codeartifact.${AWS_DEFAULT_REGION}.amazonaws.com/pypi/${CODEARTIFACT_REPO}/simple/"

In your Dockerfile, add an ARG PIP_INDEX_URL line just above your RUN pip install -r requirements.txt so it can become an environment variable during the build process:

Dockerfile

# this needs to be added before your pip install line!
ARG PIP_INDEX_URL

RUN pip install -r requirements.txt

Finally, we build the image with the PIP_INDEX_URL build-arg.

buildspec.yml

  build:
    commands:
      - echo Building the Docker image...
      - docker build -t "${IMAGE_REPO_NAME}" --build-arg PIP_INDEX_URL .

As an aside, adding ARG PIP_INDEX_URL to your Dockerfile shouldn't break any existing CI or workflows. If --build-arg PIP_INDEX_URL is omitted when building an image, pip will still use the default PyPI index.

Specifying --build-arg PIP_INDEX_URL=${PIP_INDEX_URL} is valid, but unnecessary. Specifying the argument name with no value will make Docker take its value from the environment variable of the same name[2].

Security note: If someone runs docker history ${IMAGE_REPO_NAME}, they can see the value of ${PIP_INDEX_URL}[3] . The token is only good for a maximum of 12 hours though, and you can shorten it to as little as 15 minutes with the --duration-seconds parameter of aws codeartifact get-authorization-token[4], so maybe that's acceptable. If your Dockerfile is a multi-stage build, then it shouldn't be an issue if you're not using ARG PIP_INDEX_URL in your target stage. docker build --secret does not seem to be supported in CodeBuild at this time.

Solution 2:[2]

So, here is how I solved this for now. Seems kinda hacky, but it works. (EDIT: we have since switched to @phistrom answer)

  1. In the prebuild, I run the command and copy ~/.config/pip/pip.conf to the current build directory:
pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      ...
      - echo Fetching pip.conf for PYPI
      - aws codeartifact --region us-east-1 login --tool pip --repository ....
      - cp ~/.config/pip/pip.conf .
  build:
    commands:
      - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
      - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
  1. Then in the Dockerfile, I COPY that file in, do the pip install, then rm it
COPY requirements.txt pkg/
COPY --chown=myuser:myuser pip.conf /home/myuser/.config/pip/pip.conf
RUN pip install -r ./pkg/requirements.txt
RUN pip install ./pkg
RUN rm /home/myuser/.config/pip/pip.conf

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 confiq
Solution 2