RedshiftDataClient BatchExecuteStatement combines all the queries in one

Question

I am using the BatchExecuteStatement command in the RedshiftDataClient (@aws-sdk/client-redshift-data 3.39.0) to execute a number of queries against redshift. This wasn't updating the database but not throwing an error either, so in order to troublshoot I used the ListStatements command to see if there was insights there as to what was going on. What I found was that all my queries were being combined into one. For example:

const redshiftQueries: string[] = ["select 1", "select 2", "select 3"];
await redshiftDataClient.send(
      new BatchExecuteStatementCommand({
        ...other.config.values,
        Sqls: redshiftQueries,
      })
    );

await redshiftDataClient.send(
        new ListStatementsCommand({
          MaxResults: 100,
          Status: StatusString.ALL,
        })
      );

returns

{
  "$metadata": {
    "httpStatusCode": 200,
    "requestId": "6c1dbf86-0bf1-4111-9f96-c300114daaaa",
    "attempts": 1,
    "totalRetryDelay": 0
  },
  "Statements": [
    {
      "CreatedAt": "2021-11-04T12:39:14.423Z",
      "Id": "0218bcbc-b775-4318-b111-65d7cd7eaaaa",
      "IsBatchStatement": true,
      "QueryString": "select 1select 2select 3",
      "Status": "FINISHED",
      "UpdatedAt": "2021-11-04T12:39:15.120Z"
    }
  ]
}
 

Looking at the documentation of ListStatements, I would expect in this case to receive the QueryStrings attribute with the array of queries, rather than the QueryString attribute with the queries concatenated.

The actual queries I need to run are of course a lot longer, and the issue is that when they get concatenated like this, only the first 100 characters of each one shows up in the concatenated string, so they have no hope of running successfully.

Any idea what I'm doing wrong?

Answer 1

I don't think you are doing anything wrong, I'm experienced the same problem in python. Maybe it's a bug.

My queries are being executed properly inside a transaction but Redshift Data API returns as a single string.

And, in my case, that messes the query on the Queries and Loads page, on Redshift console, I can only see the last one of my array of queries.