AWS API gateway response body template mapping (foreach)

Question

I am trying to save data in S3 through firehose proxied by API gateway. I have create an API gateway endpoint that uses the AWS service integration type and PutRecord action for firehose. I have the mapping template as

{
"DeliveryStreamName": "test-stream",
"Records": [
#foreach($elem in $input.path('$.data'))
{
"Data": "$elem"
}
#if($foreach.hasNext),#end
#end
]
}

Now when I test the endpoint with below JSON

{ 
"data": [ 
{"ticker_symbol":"DemoAPIGTWY","sector":"FINANCIAL","change":-0.42,"price":50.43},{"ticker_symbol":"DemoAPIGTWY","sector":"FINANCIAL","change":-0.42,"price":50.43} 
]
}

JSON gets modified and shows up as below after the transformation

{ticker_symbol=DemoAPIGTWY, sector=FINANCIAL, change=-0.42, price=50.43}

: is being converted to = which is not a valid JSON

Not sure if something is wrong in the above mapping template

Answer 1

The problem is, that $input.path() returns a json object and not a stringified version of the json. You can take a look at the documentation here.

The Data property expects the value to be a string and not a json object. So long story short - currently there is no built in function which can revert a json object into its stringified version. This means you need to re read the current element in the loop via $input.json(). This will return a json string representation of the element, which you then can add as Data.

Take a look at the answer here which illustrates this concept.

In your case, applying the concept described in the link above would result in a mapping like this:

    {
      "DeliveryStreamName": "test-stream",
      "Records": [
        #foreach($elem in $input.path('$.data'))
        {
          #set($json = $input.json("$[$foreach.index]"))
          "Data":"$util.base64Encode($json)",
        }
        #if($foreach.hasNext),#end
        #end
      ]
    }

Answer 2

API Gateway considers the payload data as a text and not as a Json unless explicitly specified.

Kinesis also expects data to be in encoded format while proxying through API Gateway.

Try the following code and this should work, wondering why the for loop has been commented in the mapping template.

Assuming you are not looping through the record set, the following solution should work for you.

    {
      "DeliveryStreamName": "test-stream",
      "Record": {
      "Data": "$util.base64Encode($input.json('$.Data'))Cg=="
                }
    }

Thanks & Regards, Srivignesh KN