AWS lambda doesn't return binary content

Question

I have an API which I'm deploying to AWS using CDK. I'm using the lambda proxy integration. I'm trying to create a new API call which will return a zip file, but I cannot get it to return binary data. Here's what I have:

This will eventually be called from a browser link, so I have my binaryMediaTypes set to */* as per this doc (I've also tried setting it to application/zip):

const api = new apigateway.RestApi(this, "cursive-api", {
    restApiName: props.apiName,
    ... etc etc ...
    binaryMediaTypes: ["*/*"]
});

My integration has contentHandling set to CONVERT_TO_BINARY:

const downloadIntegration = new apigateway.LambdaIntegration(downloadFn,
    {
        proxy: true,
        contentHandling: apigateway.ContentHandling.CONVERT_TO_BINARY
    });

I'm hitting this with curl and passing the Accept header:

curl -i -H "Accept: application/zip" <url>

I can see in CloudWatch that the accept header is making it through CloudFront ok:

Field           Value
@ingestionTime  1620087566765
@log            592838288812:API-Gateway-Execution-Logs_xxx/prod
@logStream      xxx
@message        (xxx) Method request headers: {Accept=application/zip, CloudFront-Viewer-Country=NZ, CloudFront-Forwarded-Proto=https, CloudFront-Is-Tablet-Viewer=false, CloudFront-Is-Mobile-Viewer=false, User-Agent=curl/7.64.1, X-Forwarded-Proto=https, CloudFront-Is-SmartTV-Viewer=false, Host=staging-api.cursive-ide.com, X-Forwarded-Port=443, X-Amzn-Trace-Id=Root=1-xxx, Via=2.0 xxx.cloudfront.net (CloudFront), X-Amz-Cf-Id=Guy-_xxx==, X-Forwarded-For=151.210.253.227, 130.176.150.98, CloudFront-Is-Desktop-Viewer=true}
@timestamp      1620087546597

I can also see in CloudWatch that I'm setting isBase64Encoded and that the content-type is right:

Field           Value
@ingestionTime  1620087566765
@log            592838288812:API-Gateway-Execution-Logs_xxx/prod
@logStream      xxx
@message        (xxx) Endpoint response body before transformations: {"statusCode":200,"headers":{"content-type":"application/zip","Content-Disposition":"attachment; filename=\"licences.zip\"","access-control-allow-origin":"https://<url>"},"isBase64Encoded":true,"body":"UEsDBETCETCETC [TRUNCATED]
@timestamp      1620087548798

However, whatever I do I only get the base64 encoded string back.