'Docker Image built on Mac OSX won't run on AWS EC2 instance

Image built on Mac OSX with M1 processor, deployed to an EC2 instance. But when scripts are run it yields the error:

standard_init_linux.go:219: exec user process caused: exec format error

Elsewhere on Stackoverflow, this is explained as a mismatch of OS architecture. Sure enough running "uname -m" on EC2 instance shows it to be x86_64, and "docker image inspect" shows the container to have architecture arm64.

Here's what I don't understand. "uname -m" on my Mac shows that to be x86_64 too. So how does the container inherit a different architecture?

More significantly, how do I build an image on my Mac that I can run on EC2?

Docker file is simply

FROM python
WORKDIR /
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src /src

with src containing, currently, some simple python scripts, executed thus:

docker run container/name python test.py

This works fine on my Mac, but gives the error above when executed on AWS.

macosdockerapple-m1apple-silicon


Solution 1:[1]

OK. Here's what's happening. My Mac has the new M1 chip and I'm running the Tech Preview version of Docker Desktop. Under the hood the chip has the arm64 architecture, but interrogating it through iTerm and VSCode it claims to be x86_64 instead, hence my confusion when I posted the question. This is probably because both those apps are being quietly run through an Intel simulator behind the scenes and that's what's responding to the uname command.

However, because the processor is really arm64, that's the base architecture when I pull Python images from Docker (I tried lots of different flavours nd version of Python - all with the same results).

To force use of an amd64 AWS-compatible image I changed the first line of the Dockerfile to:

FROM --platform=linux/x86-64 python.

When containers from this image are run on the Mac that causes a warning

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

but it's just that, a warning, and the script runs (presumably by redirecting back through the Intel simulator. The scripts now run without problem (or warning) on the EC2 instance.

Solution 2:[2]

I'm not sure why you're getting this error, but there is a nice way to get around it if you'd like and if you don't mind your code and images being public. I'm guessing that this is just home-stuff anyway, so it might not be too bad.

  1. Put your code in github.
  2. Configure a repository on hub.docker.com for your image and configure automatic builds from github
  3. ssh onto your ec2 instance and pull your image directly from docker hub

An alternative is to start with step 1, then log into your ec2 using ssh and clone the repo on that machine. You can then build it directly on a real linux machine (your osx machine doesn't run Linux, which is an instant mismatch with docker). If you build it on the server you should be able to run it there with no problems.

Solution 3:[3]

Try to run with CMD ["lscpu"] or something related like cat /proc/cpuinfo in the container, compare architectures

Another thing: you might be pulling arm architecture of python image when building, and try to run it on x86_64 (EC2)

Solution 4:[4]

In addition to what has been shared above, you could also Build a multi-arch image with Buildx.

Basically, The recent Docker versions come with a CLI command called buildx. You can use the buildx command on Docker Desktop for Mac and Windows to build multi-arch images, link them together with a manifest file, and push them all to a registry using a single command.

Here is what works for me:

Create a new builder which gives access to the new multi-architecture features.

docker buildx create --name mybuilder --use

Build the Dockerfile with buildx, passing the list of architectures to build for:

 docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t username/demo:latest --push .
 => pushing layers                                                             2.7s
 => pushing manifest for docker.io/username/demo:latest                       2.2

Where, username is a valid Docker username.

Notes: The --platform flag informs buildx to generate Linux images for AMD 64-bit, Arm 64-bit, and Armv7 architectures.
The --push flag generates a multi-arch manifest and pushes all the images to Docker Hub.

To inspect the image use the below command

docker buildx imagetools inspect username/demo:latest

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1
Solution 2 Software Engineer
Solution 3 Telinov Dmitri
Solution 4 eliarms

Related Questions