How to define parallelism for MS hosted agents with 2 pipelines

Question

We've got two YAML Pipelines, pull-request.yml and main.yml. As the names suggest, pull-request.yml runs on every PR, and main.yml runs once deployed to main. I've configured two MS hosted parallel jobs.

In main.yml using deployment jobs, I'm deploying to various Environments. It all works well, except when main.yml is executed twice, in parallel. Then, it will deploy to the same environment in each pipeline, causing issues with our IAC scripts.

Looking at the documentation, it doesn't seem possible to restrict this behavior with YAML pipelines.

My workaround now is to switch back to 1 parallel job, but I want to have to have parallel jobs for my pull-request.yml pipelines. Then, I thought, let's create another Agent Pool, but that only allows me to add self hosted agents. I want to avoid that as MS hosted agents are very convenient.

How can I have parallel jobs for my pull-request.yml but only a single instance for main.yml with MS hosted agents only?

Answer 1

It's not supported to have parallel jobs for one pull-request.yml but single parallel for another main.yaml with MS hosted agent, since Microsoft will auto detect the agent for pipeline if the requirements meet and use it to run the job.

But for your main.yml which deploying to environment, maybe you can use "Exclusive deployment lock policy" on the environment.

As doc mentioned:

With this update, you can ensure that only a single run deploys to an environment at a time. By choosing the "Exclusive lock" check on an environment, only one run will proceed. Subsequent runs which want to deploy to that environment will be paused. Once the run with the exclusive lock completes, the latest run will proceed. Any intermediate runs will be canceled.