Docker : compose file is incompatible with Amazon ECS

Question

I am trying to deploy my docker image in AWS ECS. I have created the ECR repository and done all required steps till pushing the image to ECS.

My docker-compose.yaml looks like this

version: '3'

services:

  djangoapp:
    image: xxxxx.dkr.ecr.ca-central-1.amazonaws.com/abc:latest #uri after pushing the image
    build: .
    volumes:
      - .:/opt/services/djangoapp/src
      - static_volume:/opt/services/djangoapp/static  # <-- bind the static volume
      - media_volume:/opt/services/djangoapp/media  # <-- bind the media volume
    networks:
      - nginx_network
      - database1_network
    depends_on:
      - database1

      
  nginx:
    image: nginx:1.13
    ports:
      - 80:5000
    volumes:
      - ./config/nginx/conf.d:/etc/nginx/conf.d
      - static_volume:/opt/services/djangoapp/static  # <-- bind the static volume
      - media_volume:/opt/services/djangoapp/media  # <-- bind the media volume
    depends_on:
      - djangoapp
    networks:
      - nginx_network

  database1:
    image: postgres:10
    env_file:
      - config/db/database1_env
    networks:
      - database1_network
    volumes:
      - database1_volume:/var/lib/postgresql/data

networks:
  nginx_network:
    driver: bridge
  database1_network:
    driver: bridge

volumes:
  database1_volume:
  static_volume:  # <-- declare the static volume
  media_volume:  # <-- declare the media volume

I am trying to run the command:

docker ecs compose  -n abc up 

And i get the following error:

WARN[0000] services.build: unsupported attribute        
WARN[0000] services.volumes: unsupported attribute      
ERRO[0000] published port can't be set to a distinct value than container port: incompatible attribute 
WARN[0000] services.volumes: unsupported attribute      
WARN[0000] services.env_file: unsupported attribute     
WARN[0000] services.volumes: unsupported attribute      
WARN[0000] networks.driver: unsupported attribute       
WARN[0000] networks.driver: unsupported attribute       
compose file is incompatible with Amazon ECS

I am using the latest version of docker i.e 19.03.08 and the latest aws-cli/2.0.39.

Answer 1

I am facing the same trouble and I was able to accomplish it by removing all attributes in docker-compose.yaml except for image.

In your djangoapp service under the image attribute, you set the value to a URI with the comment "the uri after pushing the image". Presumably this is the URI to a locally-built docker image of djangoapp, which was pushed to a ECR repository.

Since you already built and pushed the djangoapp image to ECR, just leave the image attribute and comment out all other attributes listed in the error message from docker-compose.yaml:

1. build
2. volumes
3. ...

In my case it helped.

The list of supported attributes: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-parameters.html

Answer 2

Docker decided to not allow for this. This is what they said on a Slack convo: "We decided to not support this as this would only apply to ingress traffic, not service-to-service, which would be both confusing and inconsistent with local development".

Answer 3

I'm not sure what you're trying to achieve, but it looks like you want to have a django service backed by a postgres database, and you want to use nginx as a reverse proxy to forward requests to django?

If you are trying to do that, then just us a single network for your cluster, and get rid of the driver option(s) for the networks. Removing those driver options will get rid of the networks.driver errors. The networks in docker compose will map to EC2 security groups. They're not like private/bridged/nat networks you would use in VM management like VMWare or Hyper-V. Putting all of your services on the same network will allow them to communicate with each other if that's what you're looking for. They will also be able to communicate out to the internet, but only those that have ports set will be reachable directly from the internet.

Regarding ports, only symmetrical port mapping is supported in ECS. This means you can't map one external port to a different internal port. They have to be the same. So your nginx configuration must use either 80, or 5000 - not both. Fixing that will get rid of your 3rd error.

As @Maksym specified, you don't need the build option under your djangoapp service. You already have the image built and are just deploying it.

For your volumes, you're trying to mount a host path with .:/opt/services/djangoapp/src. When you use docker compose to deploy to ECS, it uses CloudFormation to deploy your stack and each of your services run "serverless", so there isn't a host to mount a path from. In your case, since you're building djangoapp image yourself, just update your Dockerfile to copy the desired contents to the /opt/services/djangoapp/src filter as part of the image build. Do the same with your nginx service - create your own nginx image and include the files you want in /etc/nginx/conf.d, push it to ECR, and then use that image in your compose yaml. One of those files should be a configuration to reverse proxy port 80 to your djangoapp:5000 port.

For your other errors, I'm not sure. The configuration for env_file looks fine, as does your named volume mappings like static_volume:/opt/services/djangoapp/static. I don't know if updating your version heading to 3.7 in the top of your file will help.

In the end, your file should look similar to this:

version: '3.7'

services:

  djangoapp:
    image: xxxxx.dkr.ecr.ca-central-1.amazonaws.com/abc:latest #uri after pushing the image
    volumes:
      - static_volume:/opt/services/djangoapp/static  # <-- bind the static volume
      - media_volume:/opt/services/djangoapp/media  # <-- bind the media volume
    networks:
      - app_network
    depends_on:
      - database1

      
  nginx:
    image: xxxxx.dkr.ecr.ca-central-1.amazonaws.com/my-nginx:latest
    ports:
      - 80
    volumes:
      - static_volume:/opt/services/djangoapp/static  # <-- bind the static volume
      - media_volume:/opt/services/djangoapp/media  # <-- bind the media volume
    depends_on:
      - djangoapp
    networks:
      - app_network

  database1:
    image: postgres:10
    env_file:
      - config/db/database1_env
    networks:
      - app_network
    volumes:
      - database1_volume:/var/lib/postgresql/data

networks:
  app_network:
    name: app_network

volumes:
  database1_volume:
    name: app_database1_volume
  static_volume:
    name: app_static_volumne
  media_volume:
    name: app_media_volume