Creating the topic rule does not create the trigger on the lambda

Question

This issue looks very much like a bug but I believe there must be something wrong in my terraform file because I can't find anybody on the web having the same problem.

Here is the part of my terraform file that creates a lambda and a topic rule for it:

resource "aws_lambda_function" "rds_persist" {
  filename         = "${local.rds_persist_file_path}"
  function_name    = "RdsPersist-${var.env}"
  role             = "${aws_iam_role.lambda_role.arn}"
  handler          = "package.handler"
  source_code_hash = "${local.rds_persist_package_hash}"
  runtime          = "nodejs8.10"
  memory_size      = 128
  timeout          = 10

  vpc_config = {
    subnet_ids         = ["${var.private_subnet_ids}"]
    security_group_ids = ["${aws_security_group.all_vpc_access.id}"]
  }

  environment {
    variables = {
      DB             = "${var.database_url}"
      IOT_DEVICE_ARN = "${var.iot_device_v1_sns_arn}"
    }
  }
}

resource "aws_iot_topic_rule" "rds_push" {
  name        = "${var.env}_RdsPush"
  description = "Pushes events to a persistence lambda (rds store)"
  enabled     = true
  sql         = "SELECT * as payload, topic() as topic, timestamp() AS timestamp FROM '#' WHERE startswith(clientid(), '${var.env}-')"
  sql_version = "2016-03-23"

  lambda {
    function_arn = "${aws_lambda_function.rds_persist.arn}"
  }
}

Here is the result in AWS Console:

If I delete and re-add the rule in the console, then the trigger appears on the lambda.

It might be the case your lambda function that your topic uses is being created before the function.

I tested this also by tainting the topic rule alone so it got recreated (see the logs below). Unfortunately it didn't solve the issue.

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

-/+ module.lambda.aws_iot_topic_rule.rds_push (tainted) (new resource required)
      id:                             "dev_RdsPush" => <computed> (forces new resource)
      arn:                            "arn:aws:iot:eu-west-1:827689093226:rule/dev_RdsPush" => <computed>
      description:                    "Pushes events to a persistence lambda (rds store)" => "Pushes events to a persistence lambda (rds store)"
      enabled:                        "true" => "true"
      lambda.#:                       "1" => "1"
      lambda.1860721139.function_arn: "arn:aws:lambda:eu-west-1:827689093226:function:RdsPersist-dev" => "arn:aws:lambda:eu-west-1:827689093226:function:RdsPersist-dev"
      name:                           "dev_RdsPush" => "dev_RdsPush"
      sql:                            "SELECT * as payload, topic() as topic, timestamp() AS timestamp FROM '#' WHERE startswith(clientid(), 'dev-')" => "SELECT * as payload, topic() as topic, timestamp() AS timestamp FROM '#' WHERE startswith(clientid(), 'dev-')"
      sql_version:                    "2016-03-23" => "2016-03-23"


Plan: 1 to add, 0 to change, 1 to destroy.

Update: I just found a very similar issue at a different place:

There's supposed to be an SNS subscription between this another lambda and a SNS. Here is the relevant code in the terraform:

resource "aws_sns_topic_subscription" "conference_call" {
  topic_arn = "${module.sns.conference_call_arn}"
  protocol  = "lambda"
  endpoint  = "${module.lambda.messaging_arn}"
}

(Obviously I checked the resources and they are correct)

In the console I don't see the trigger in the lambda but I do see the subscription in SNS:

Update: exact same issue when creating the resources using AWS CLI

# For the first issue
$ aws iot create-topic-rule --rule-name dev_RdsPush --topic-rule-payload '{"sql":"SELECT * as payload, topic() as topic, timestamp() AS timestamp FROM \'#\' WHERE startswith(clientid(), \'dev-\')","actions":[{"lambda":{"functionArn":"arn:aws:lambda:eu-west-1:xxxxxxxxx:function:RdsPersist-dev"}}]}'

# For the second issue
$ aws sns subscribe --topic-arn arn:aws:sns:eu-west-1:xxxxxxxx:conference-call-v1-dev --protocol lambda --notification-endpoint arn:aws:lambda:eu-west-1:xxxxxxxxx:function:Messaging-dev

---

Solution:

Add these:

IoT:

resource "aws_lambda_permission" "conference_call_sns" {
  statement_id  = "AllowExecutionFromSNS"
  action        = "lambda:InvokeFunction"
  function_name = "${aws_lambda_function.conference_call.function_name}"
  principal     = "sns.amazonaws.com"
  source_arn    = "${var.conference_call_sns_arn}"
}

SNS:

resource "aws_lambda_permission" "messaging_sns" {
  statement_id  = "AllowExecutionFromSNS"
  action        = "lambda:InvokeFunction"
  function_name = "${aws_lambda_function.messaging.function_name}"
  principal     = "sns.amazonaws.com"
  source_arn    = "${var.conference_call_sns_arn}"
}

Answer 1

You need to add a lambda permission to allow IoT to invoke lambda. The Lambda console uses the permissions of the function to show what can invoke it.

https://docs.aws.amazon.com/iot/latest/developerguide/iot-rule-actions.html#lambda-rule.

Answer 2

It might be the case your lambda function that your topic uses is being created before the function. Try adding depends_on = ["aws_lambda_function.rds_persist"] on your aws_iot_topic_rule and see how it goes.

Answer 3

In my case this was the missing permission:

resource "aws_lambda_permission" "rds_topic_rule_permission" {
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.rds_persist.function_name
  principal     = "iot.amazonaws.com"
  source_arn    = aws_iot_topic_rule.rds_push.arn
}

See: https://github.com/hashicorp/terraform-provider-aws/issues/24196#issuecomment-1103979956