'How to use AWS CodeArtifact *within* A Dockerfile in AWSCodeBuild
I am trying to do a pip install from codeartifact from within a dockerbuild in aws codebuild.
This article does not quite solve my problem: https://docs.aws.amazon.com/codeartifact/latest/ug/using-python-packages-in-codebuild.html
The login to AWS CodeArtifct is in the prebuild; outside of the Docker context.
But my pip install
is inside my Dockerfile (we pull from a private pypi registry).
How do I do this, without doing something horrible like setting an env variable to the password derived from reading ~/.config/pip.conf/
after running the login command in prebuild?
Solution 1:[1]
You can use the environment
variable: PIP_INDEX_URL
[1].
Below is an AWS CodeBuild buildspec.yml
file where we construct the
PIP_INDEX_URL
for CodeArtifact by using
this example from the AWS documentation.
buildspec.yml
pre_build: commands: - echo Getting CodeArtifact authorization... - export CODEARTIFACT_AUTH_TOKEN=$(aws codeartifact get-authorization-token --domain "${CODEARTIFACT_DOMAIN}" --domain-owner "${AWS_ACCOUNT_ID}" --query authorizationToken --output text) - export PIP_INDEX_URL="https://aws:${CODEARTIFACT_AUTH_TOKEN}@${CODEARTIFACT_DOMAIN}-${AWS_ACCOUNT_ID}.d.codeartifact.${AWS_DEFAULT_REGION}.amazonaws.com/pypi/${CODEARTIFACT_REPO}/simple/"
In your Dockerfile, add an ARG PIP_INDEX_URL
line just above
your RUN pip install -r requirements.txt
so it can become an environment
variable during the build process:
Dockerfile
# this needs to be added before your pip install line! ARG PIP_INDEX_URL RUN pip install -r requirements.txt
Finally, we build the image with the PIP_INDEX_URL
build-arg.
buildspec.yml
build: commands: - echo Building the Docker image... - docker build -t "${IMAGE_REPO_NAME}" --build-arg PIP_INDEX_URL .
As an aside, adding ARG PIP_INDEX_URL
to your Dockerfile shouldn't break any
existing CI or workflows. If --build-arg PIP_INDEX_URL
is omitted when
building an image, pip will still use the default PyPI index.
Specifying --build-arg PIP_INDEX_URL=${PIP_INDEX_URL}
is valid, but
unnecessary. Specifying the argument name with no value will make Docker take
its value from the environment variable of the same
name[2].
Security note: If someone runs docker history ${IMAGE_REPO_NAME}
, they can
see the value
of ${PIP_INDEX_URL}
[3]
. The token is only good for a maximum of 12 hours though, and you can shorten
it to as little as 15 minutes with the --duration-seconds
parameter
of aws codeartifact get-authorization-token
[4],
so maybe that's acceptable. If your Dockerfile is a multi-stage build, then it
shouldn't be an issue if you're not using ARG PIP_INDEX_URL
in your target
stage. docker build --secret
does not seem to be supported in CodeBuild at this time.
Solution 2:[2]
So, here is how I solved this for now. Seems kinda hacky, but it works. (EDIT: we have since switched to @phistrom answer)
- In the prebuild, I run the command and copy
~/.config/pip/pip.conf
to the current build directory:
pre_build:
commands:
- echo Logging in to Amazon ECR...
...
- echo Fetching pip.conf for PYPI
- aws codeartifact --region us-east-1 login --tool pip --repository ....
- cp ~/.config/pip/pip.conf .
build:
commands:
- docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
- docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
- Then in the Dockerfile, I
COPY
that file in, do thepip install
, thenrm it
COPY requirements.txt pkg/
COPY --chown=myuser:myuser pip.conf /home/myuser/.config/pip/pip.conf
RUN pip install -r ./pkg/requirements.txt
RUN pip install ./pkg
RUN rm /home/myuser/.config/pip/pip.conf
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | confiq |
Solution 2 |