ECS - target type ip is incompatible with the bridge network mode specified in the task definition

Question

Problem

Getting target type ip,which is incompatible with the bridge network mode error when trying to create an ECS service with an existing NLB whose target group uses IP as the target type.

The error is from Terraform as using it to create all the AWS resources.

Error: InvalidParameterException: The provided target group arn:aws:elasticloadbalancing:$REGION:$ACCOUNT:targetgroup ... has target type ip,which is incompatible with the bridge network mode specified in the task definition.

The TF_DEBUG output of the Terraform Github issue #11719 seems to be indicating it is the restriction, if Terraform (or its message) is correct.

2020-01-22T20:04:46.819Z [DEBUG] plugin.terraform-provider-aws_v2.45.0_x4: 2020/01/22 20:04:46 [DEBUG] [aws-sdk-go] {"__type":"InvalidParameterException","message":"The provided target group arn:aws:elasticloadbalancing:us-east-1:xxx:targetgroup/llprd20200122052638603300000006/a0a2d775807f6620 has target type ip, which is incompatible with the bridge network mode specified in the task definition."}

Question

Please advise if this can be a limitation of AWS. As far as I looked into the AWS documentation so far, there is no information that IP target type cannot be used for bridge network mode. However, would like to make 100% sure.

• ECS Service - Creating a Network Load Balancer - Configure Routing

  4. For Target type, choose whether to register your targets with an instance ID or an IP address.
     
     Important
     If your service's task definition uses the awsvpc network mode (which is required for the Fargate launch type), you must choose ip as the target type, not instance. This is because tasks that use the awsvpc network mode are associated with an elastic network interface, not an Amazon EC2 instance.
     
     You cannot register instances by instance ID if they have the following instance types: C1, CC1, CC2, CG1, CG2, CR1, G1, G2, HI1, HS1, M1, M2, M3, and T1. You can register instances of these types by IP address.

Terraform

• Terraform aws_lb_target_group

resource "aws_lb_target_group" "this" {
  count = length(var.listeners)
  name_prefix           = "${substr("${var.name}", 0, 6)}"
  vpc_id                = "${var.vpc_id}"
  target_type           = "ip"
  port                  = 8080
  protocol              = "tcp"
  ...
}

I did not specify the network_mode in the aws_ecs_task_definition resource configuration, so default "bridge" is used.

TF_DEBUG

...
2020-03-03T18:54:10.301+1100 [DEBUG] plugin.terraform-provider-aws_v2.50.0_x4: 2020/03/03 18:54:10 [DEBUG] [aws-sdk-go] {"__type":"InvalidParameterException","message":"The provided target group arn:aws:elasticloadbalancing:us-east-2:ACCOUNT:targetgroup/****/4689fc19ff99ca57 has target type ip, which is incompatible with the bridge network mode specified in the task definition."}
2020-03-03T18:54:10.301+1100 [DEBUG] plugin.terraform-provider-aws_v2.50.0_x4: 2020/03/03 18:54:10 [DEBUG] [aws-sdk-go] DEBUG: Validate Response ecs/CreateService failed, attempt 0/25, error InvalidParameterException: The provided target group arn:aws:elasticloadbalancing:us-east-2:ACCOUNT:targetgroup/****/4689fc19ff99ca57 has target type ip, which is incompatible with the bridge network mode specified in the task definition.
...

Environment

• ECS type is EC2, not Fargate
• Using Terraform v0.12.20 running on Ubuntu "18.04.4 LTS (Bionic Beaver)"

Answer 1

As stated in the AWS service discovery guidelines, you cannot reference ECS containers with bridge network mode using an ip. In fact, you can only specify SRV DNS records for this kind of services.

Options here are either changing the task definition network mode to awsvpc or changing the target_type to instance.

Personally I have had only experiences with awsvpc network mode.

Answer 2

I had to migrate some ECS services using Fargate to an ECS EC2 Cluster. The bridge network mode is a must because when we tried using awsvpc, we could only place 3 containers per EC2 instance, as awsvpc network mode will attach an ENI (Elastic Network Interface) to each task. You can only have 4 ENIs per EC2 instance (depending on the instance type), so the cluster was over-provisioning instances to place the service tasks.

Important: I'm using the bridge network mode with dynamic ports (by not specifying a hostPort in the portMappings section of your containerDefinitions).

aws_lb_target_group resource configuration:

• target_type is set to instance
• the port parameter inside the health_check block is omitted (it will be automatically set to Traffic port)

Example:

resource "aws_lb_target_group" "service_bridge" {
  port                 = 3000 # Service traffic port
  protocol             = "HTTP"
  target_type          = "instance"
  vpc_id               = "vpc-123"
  deregistration_delay = 300

  health_check {
    healthy_threshold   = 3
    unhealthy_threshold = 3
    interval            = 30
    matcher             = "200-299"
    path                = "/"
    protocol            = "HTTP"
  }
}

The target group will have the EC2 cluster instances with the task registered but redirecting the traffic to the task randomly assigned port.

Target Group details (imgur link)

Registered targets (imgur link)

Health check settings (imgur link)