Resolve Azure YAML Pipeline overlapping variable names in multiple variable groups

Question

We're working on converting our Classic Azure Pipelines to YAML Pipelines. One thing that is not clear is how to ensure that two different variable groups with variables with the same name but different meaning don't step on each other.

For example, if I have variable groups vg1 and vg2, each with variable named secretDataDestination, how do I ensure that the correct secretDataDestination is used in a YAML Pipeline?

A more concerning example is, if we initially have two variable groups without overlapping variable names, how do we ensure that adding a newly-overlapping variable name to a group doesn't replace use of the variable as originally intended?

Answer 1

A workaround is leveraging output variables in Azure DevOps with some small inline PowerShell task code.

First, create 2 jobs. Each job with their own variable group, in this case Staging and Prod. Both groups contain the variables apimServiceName and apimPrefix. Add the variables as a job output by echoing them as isOutput=true like this:

      - job: StagingVars
        dependsOn:
        variables:
          - group: "Staging"
        steps:
          - powershell: >-
              echo "##vso[task.setvariable variable=apimServiceName;isOutput=true]$(apimServiceName)"
              echo "##vso[task.setvariable variable=apimPrefix;isOutput=true]$(apimPrefix)"
            name: setvarStep

      - job: ProdVars
        dependsOn:
        variables:
          - group: "Prod"
        steps:
          - powershell: >-
              echo "##vso[task.setvariable variable=apimServiceName;isOutput=true]$(apimServiceName)"
              echo "##vso[task.setvariable variable=apimPrefix;isOutput=true]$(apimPrefix)"
            name: setvarStep

Then, use the variables in a new job, where you specify a new variable name and navigate to the job output to get a value, this works because the variable groups are each placed into their own job, so they will not overwrite any variable:

      - job:
        dependsOn:
          - StagingVars
          - ProdVars
        variables:
          ServiceNameSource: "$[ dependencies.StagingVars.outputs['setvarStep.apimServiceName'] ]"
          UrlprefixSource: "$[ dependencies.StagingVars.outputs['setvarStep.apimPrefix'] ]"
          ServiceNameDestination: "$[ dependencies.ProdVars.outputs['setvarStep.apimServiceName'] ]"
          UrlprefixDestination: "$[ dependencies.ProdVars.outputs['setvarStep.apimPrefix'] ]"

Answer 2

if I have variable groups vg1 and vg2, each with variable named secretDataDestination, how do I ensure that the correct secretDataDestination is used in a YAML Pipeline?

Whether we use classic mode or YAML, it is not recommended to define a variable with the same name in different variable groups. Because when you refer to different variable groups containing the same variable name in the same pipeline, you cannot avoid step on each other.

When you use the same variable name in different variable group in the same pipeline, just like Matt said,

"You can reference multiple variable groups in the same pipeline. If multiple variable groups include the same variable, the variable group included last in your YAML file will set the variable's value."

variables:
- group: variable-group1
- group: variable-group2

That means that the variable value in the variable group written later will overwrite the variable value in the variable group written first

I guess you already know this, so you post your second question. Let us now turn to the second question.

if we initially have two variable groups without overlapping variable names, how do we ensure that adding a newly-overlapping variable name to a group doesn't replace use of the variable as originally intended?

Indeed, Azure devops currently does not have such a function or mechanism to intelligently detect whether different variable groups have the same variable name, and give a prompt.

I think this is a reasonable request, I add your request for this feature on our UserVoice site which is our main forum for product suggestions:

The ability to detect the same variable in a variable group

As workaround, the simplest and most direct way is that open the variable group of your pipeline link in the Library tab, and directly ctrl + F to search for the existence of the same variable.

Another way is to use REST API Variablegroups - Get Variable Groups By Id to get all the variables, then the loop compares with the variable we are going to enter whether the same variable exists.