API Gateway Proxy for S3 with subdirectories

Question

I created an API Gateway method GET with integration type "AWS Service" for "S3". I defined a Path override: {object} that is mapped from method.request.path.item

As long as the path contains only the bucket itself (./bucket/mybucketname), it works and the value of {item} is mybucketname.

But when i specify for example ../bucket/mybucketname/foo/bar/test.txt as path the {item} value should be mybucketname/foo/bar/test.txt but it is empty. I think it's because of the nested path

When i choose catch-all path variables {proxy+} I'm not able to choose integration type "AWS Service" any longer.

Any hints what I'm doing wrong or what i should do in order to retrieve files/folders in subfolders from S3 via API Gateway?

Answer 1

This can be done using proxy resource.

Here are my steps to achieve what @Johnny90 wants:

1. Click on your API in the API Gateway console.

2. Choose the /bucket resource and then choose Create Resource from the Actions drop-down menu. Then, do the following in the New Child Resource pane.

   a. Tick "Configure as proxy resource".

   b. Use the default proxy for Resource Name.

   c. Use {proxy+} for Resource Path.

   d. Choose Create Resource.

3. Choose HTTP Proxy for Integration type and type the Endpoint URL as any website (e.g., https://my-website.com/{proxy} ). Then choose Save.

4. Choose Integration Request

   a. Select AWS Service as the Integration type.

   b. From the AWS Region drop-down list, choose the region that your S3 bucket resides in.

   c. From AWS Service, choose S3. For AWS Subdomain, leave it blank.

   d. For HTTP method, choose GET.

   e. Choose Use path override for Action Type. and type bucket/{proxy}.

   f. Paste an IAM role that has enough permissions.

   g. Click Save.

After changing the integration request from HTTP Proxy integration to AWS Service, we have to add some settings for the API. First, you must set up the URL Path Parameters so that API Gateway can understand the {proxy} variable defined in resource path in Integration Request.

1. Extend URL Path Parameters in Integration Request and then choose Add path.

2. Type proxy in the Name column and method.request.path.proxy in the Mapped from column.

Second, choose Method Response from Method Execution.

1. Choose Add Response. Type 200 for HTTP status.

2. Expand the response of the 200 code.

   a. Choose Add Header. Type Content-Type for the Name.

   b. Click Add Response Model. Type application/json for Content type and choose Empty from the Models drop-down menu.

Finally, choose Integration Response from Method Execution.

1. Extend the 200 Method response status

2. For Header Mapping, you should see the Content-Type in the Response header column. Type integration.response.header.Content-Type in the Mapping value column.

The following are my configurations:

Integration Request:

Integration Response:

Method Response:

Answer 2

The key is how you pass the value for {object} variable in Path Override. I think you set url path parameter for integration request from request path, like this:

object = method.request.path.object     

I believe the problem was because the object has "/" in it, so does the api path (resource). It cause the api gateway can only recognized object in root path.

What I do is set url path parameter for integration request from request querystring like this:

object = method.request.querystring.object

It works

Or you can try encode the object path using url encode, replace '/' with '%2F'. I never tried though.

Answer 3

Path override : bucket_name/{folder} pattern as per level, it's for first {folder}.

you can access s3 nested path like this.

this is for last {object}, so mentioned all path map. this link will helpfull https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-s3.html

Answer 4

@johnny90, First configure as lambda proxy while creating resource {proxy+}...After creating..update integration type as "AWS Service".Then you can configure any AWS service to your API gateway.

Answer 5

You might use "%2F" as path separator in your requesting URL, e.g.:

foo%2Fbar in the URL will be translated into foo/bar S3 path.