'Error connecting to EFS mount from AWS ECS fargate container?

I'm trying to setup via CloudFormation an EFS mount for self-hosted Prometheus. Below is the CloudFormation for my setup:

Resources:
  ServiceSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: 'Prometheus SG'
      VpcId:
        Fn::ImportValue: !Sub '${NetworkStackName}-VPCID'
      SecurityGroupIngress:
        # Allow access from the Load Balancer only
        - SourceSecurityGroupId:
            Fn::ImportValue: !Sub '${LBStackName}-SG-LB'
          IpProtocol: tcp
          FromPort: 9090
          ToPort: 9090
      Tags:
        - Key: Name
          Value: !Sub 'SG-Prometheus-LB-${Stage}'

  EFSSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: 'Prometheus EFS SG'
      VpcId:
        Fn::ImportValue: !Sub '${NetworkStackName}-VPCID'
      SecurityGroupIngress:
        # Allow access from the ECS Service only
        - SourceSecurityGroupId: !Ref ServiceSecurityGroup
          IpProtocol: tcp
          FromPort: 2049
          ToPort: 2049
      Tags:
        - Key: Name
          Value: !Sub 'SG-Prometheus-EFS-${Stage}'

  MountTarget1:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId:
        Fn::ImportValue: !Sub '${DataStackName}-EFSID'
      SecurityGroups:
        - !Ref EFSSecurityGroup
      SubnetId:
        Fn::ImportValue: !Sub '${NetworkStackName}-PRIVATE-SUBNET-A1'

  MountTarget2:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId:
        Fn::ImportValue: !Sub '${DataStackName}-EFSID'
      SecurityGroups:
        - !Ref EFSSecurityGroup
      SubnetId:
        Fn::ImportValue: !Sub '${NetworkStackName}-PRIVATE-SUBNET-B1'

  Prometheus:
    Type: AWS::ECS::Service
    DependsOn: ListenerRule
    Properties:
      Cluster: !Ref Cluster
      LaunchType: FARGATE
      DesiredCount: !FindInMap [ ECSTaskDefinition, !Ref Stage, DesiredTaskCount ]
      TaskDefinition: !Ref TaskDefinition
      HealthCheckGracePeriodSeconds: 300
      DeploymentConfiguration:
        MaximumPercent: 200
        MinimumHealthyPercent: 100
      LoadBalancers:
        - ContainerName: 'Prometheus-Container'
          ContainerPort: 9090
          TargetGroupArn: !Ref TargetGroup
      NetworkConfiguration:
        AwsvpcConfiguration:
          SecurityGroups:
            - !Ref ServiceSecurityGroup
          Subnets:
            - Fn::ImportValue: !Sub '${NetworkStackName}-PRIVATE-SUBNET-A1'
            - Fn::ImportValue: !Sub '${NetworkStackName}-PRIVATE-SUBNET-B1'
      EnableECSManagedTags: true

  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: Prometheus
      RequiresCompatibilities:
        - FARGATE
      Cpu: !FindInMap [ ECSTaskDefinition, !Ref Stage, CPU ]
      Memory: !FindInMap [ ECSTaskDefinition, !Ref Stage, Memory ]
      ContainerDefinitions:
        - Name: 'Prometheus-Container'
          Essential: true
          Image: !Sub '${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${RepositoryName}:${ImageTag}'
          Cpu: !FindInMap [ ECSTaskDefinition, !Ref Stage, CPU ]
          Memory: !FindInMap [ ECSTaskDefinition, !Ref Stage, Memory ]
          PortMappings:
            - ContainerPort: 9090
          MountPoints:
            - SourceVolume: 'Prometheus-Volume'
              ContainerPath: '/prometheus'
              ReadOnly: false
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group:
                Fn::ImportValue: !Sub '${DataStackName}-CW-LogsGroup'
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: 'PrometheusApp'
      Volumes:
        - Name: 'Prometheus-Volume'
          EFSVolumeConfiguration:
            FilesystemId:
              Fn::ImportValue: !Sub '${DataStackName}-EFSID'
            RootDirectory: "/"
            TransitEncryption: ENABLED
      NetworkMode: awsvpc
      TaskRoleArn:
        Fn::ImportValue: !Sub '${LBStackName}-TaskRoleArn'
      ExecutionRoleArn: !GetAtt ExecutionRole.Arn

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      VpcId:
        Fn::ImportValue: !Sub '${NetworkStackName}-VPCID'
      TargetType: ip
      Port: 9090
      Protocol: HTTP
      Matcher:
        HttpCode: '200'
      TargetGroupAttributes:
        - Key: 'deregistration_delay.timeout_seconds'
          Value: '60'
      HealthCheckIntervalSeconds: 10
      HealthCheckPath: /status
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 2
      HealthyThresholdCount: 2

  ListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      ListenerArn:
        Fn::ImportValue: !Sub '${LBStackName}-LB-LISTENER'
      Priority: 2
      Conditions:
        - Field: path-pattern
          Values:
            - /*
      Actions:
        - TargetGroupArn: !Ref TargetGroup
          Type: forward

  TaskPolicy:
    Type: AWS::IAM::Policy
    Properties:
      PolicyName: 'Prometheus-TaskPolicy'
      Roles:
        - Fn::ImportValue: !Sub '${LBStackName}-TaskRole'
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: EnablePutMetricData
            Effect: Allow
            Resource: '*'
            Action:
              - cloudwatch:PutMetricData

  ExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ["ecs-tasks.amazonaws.com"]
            Action:
              - sts:AssumeRole
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'

  Cluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Sub "Prometheus-${Stage}"

Everything gets deployed successfully but the containers die with the error message: ResourceInitializationError: failed to invoke EFS utils commands to set up EFS volumes: stderr: Failed to resolve "fs-xxxxxxxxxxx.efs.us-east-1.amazonaws.com. I've checked https://aws.amazon.com/premiumsupport/knowledge-center/fargate-unable-to-mount-efs/ and don't think we have an issue with zones. Any ideas welcome.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source