Lambda function exclude .csv.metadata files from the output

Question

I have a Lambda function that executing Athena query and exporting the output in csv file to S3 bucket. Now in my S3 bucket I'm getting 2 files .csv and .csv.metadata

My questions is how can I exclude the .csv.metadata files?

import boto3


def lambda_handler(event, context):
    query_1 =   "<MY-QUERY-HERE>"
                 
    database = "<MY-DB-HERE>"
    s3_output = "MY-S3-BUCKET"

    client = boto3.client('athena')

    response = client.start_query_execution(QueryString = query_1,
                                        QueryExecutionContext={
                                            'Database': database
                                        },
                                        ResultConfiguration={
                                            'OutputLocation': '<MY-S3-BUCKET>'
                                        }
                                        )
    return response

Answer 1

Added this to my S3 bucket policy, and it works like a charm. Doing exactly what i need. And you can chose any prefix/bucket you want.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "StatementPrefixDeny",
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::prefix/*.csv.metadata"
        }
    ]
}

Answer 2

If you don't need that file, you can add 1 more line to delete that file in S3 bucket.

EDIT: you can try this:

s3 = boto3.resource('s3')

s3.Object('your-bucket', 'your-key').delete()

Cause according to the document, we don't have any options to change the result items:

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena.html#Athena.Client.start_query_execution