'Azure Stream Analytics CreateOrReplace Transformation Conflict or Bad Request

We are currently streaming data from an Azure Event Hub via Azure Stream Analytics to Cosmos DB. I want to be able to programatically change the query (/ Transformation) that is completed as the data is streamed.

var existingQuery = AnalyticsClient.Transformations.Get(handler.ResourceGroup, handler.JobName, handler.JobName);

if (job.JobState == "Started")
        {
            AnalyticsClient.StreamingJobs.BeginStopAsync(handler.ResourceGroup, handler.JobName).Wait();
        }
        var transformation = new Transformation()
        {
            Query = $@"SELECT 
                        [key] as partition_key
                    INTO[{outputName}]
                    FROM Input",
            StreamingUnits = 1
        }; 
AnalyticsClient.Transformations.CreateOrReplace(transformation, handler.ResourceGroup, handler.JobName, handler.JobName);

Stream Analytics Job Properties - Transformations shows as null.

It is unclear what the default name of the transformation might be but trying to get a transformation (of the same name as the job) from a job:

{
    "code": "NotFound",
    "message": "StreamAnalytics_Prototype_2 does not exist in Stream Analytics job 'StreamAnalytics_Prototype_2' in resource group 'removed' in subscription 'removed'.",
    "details": {
        "code": "404",
        "message": "StreamAnalytics_Prototype_2 does not exist in Stream Analytics job 'StreamAnalytics_Prototype_2' in resource group 'removed' in subscription 'removed'.",
        "correlationId": "removed",
        "requestId": "removed"
    }
}

Trying to create a transformation:

{
    "code": "BadRequest",
    "message": "The number of transformations defined for this job exceeds the maximum limit of 1 that is supported.",
    "details": {
        "code": "400",
        "message": "The number of transformations defined for this job exceeds the maximum limit of 1 that is supported."
    }
}
azureazure-cosmosdbazure-eventhubazure-stream-analytics


Solution 1:[1]

If you create the transformation from Azure portal, the default name is for the transformation "Transformation". If you create the transformation from SDK, you need to specific the name in code.

streamAnalyticsManagementClient.Transformations.CreateOrReplace(transformation, resourceGroupName, streamingJobName, transformationName);

By the way, I found the default transformation name from Activity log after added a query from Azure portal.

enter image description here

Solution 2:[2]

You can use Powershell or CloudPowershell to get the transformation name by using Get-AzStreamAnalyticsJob. In my case, there were multiple jobs and I needed to find the transformation name for the first job in the list.

$ (Get-AzStreamAnalyticsJob -expand 'Transformation')[0].TransformationName

I couldn't find the transformation name in Azure portal though.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Amor
Solution 2 Jeremy Caney

Related Questions