Job as a variable and use variable inside this job | github action & yaml

Question

I'm currently deploying a security tool for my cluster. It worked well but I want to reduce the length of the code and avoid repeating code inside the file.

Here is the situation:

on:
  pull_request:
    path:
      - 'ionos/terraform/dev/*.tf'
      - 'ionos/terraform/prod/*/*/*.tf'

jobs:
  # JOB to run change detection
  changes:
    runs-on: ubuntu-latest
    # Set job outputs to values from filter step
    outputs:
      Ionos_dev: ${{ steps.filter.outputs.Ionos_dev }}
      Ionos_prod: ${{ steps.filter.outputs.Ionos_prod }}
    steps:
    # For pull requests it's not necessary to checkout the code
    - uses: dorny/paths-filter@v2
      id: filter
      with:
        filters: |
          Ionos_dev:
            - 'ionos/terraform/dev/*.tf'
          Ionos_prod:
            - 'ionos/terraform/prod/*/*/*.tf'

Duplicated part

  Ionos_prod:
    name: tfsec sarif report ionos_prod
    needs: changes
    if: ${{ needs.changes.outputs.Ionos_prod == 'true' }}
    runs-on: ubuntu-latest
    steps:
      - name: Clone repo
        uses: actions/checkout@master

      - name: tfsec sarif ionos_dev
        uses: aquasecurity/[email protected]
        with:
          working_directory: ionos/terraform/prod/
          sarif_file: tfsec.sarif

      - name: Upload SARIF file
        uses: github/codeql-action/upload-sarif@v1
        with:
          sarif_file: tfsec.sarif

Ionos_dev:
    name: tfsec sarif report ionos_dev
    needs: changes
    if: ${{ needs.changes.outputs.Ionos_dev == 'true' }}
    runs-on: ubuntu-latest
    steps:
      - name: Clone repo
        uses: actions/checkout@master

      - name: tfsec sarif ionos_dev
        uses: aquasecurity/[email protected]
        with:
          working_directory: ionos/terraform/dev/
          sarif_file: tfsec.sarif

      - name: Upload SARIF file
        uses: github/codeql-action/upload-sarif@v1
        with:
          sarif_file: tfsec.sarif

I have more than 2 duplicated jobs, that's why I want to make the job as a variable.

My problem is I don't see how can I create the job as a variable and pass these two variables inside the job just created:

if: ${{ needs.changes.outputs.Ionos_prod == 'true' }}

&

working_directory: ionos/terraform/prod/

Anny suggestions?

Answer 1

After few days of research, and based on that documentation (I haven't found it before): https://docs.github.com/pt/actions/using-jobs/using-a-matrix-for-your-jobs

I finally solved my problem.

Here is the code and some explanations at the end of it.

 on:
  pull_request:
    types: [synchronize, reopened, labeled]
    paths:
      - 'aws/dns/domains/**'
      - 'ionos/terraform/prod/**'
      - 'ionos/terraform/dev/**'
      - 'azure/terraform/**'

jobs:
  changes:
    runs-on: ubuntu-latest
    #Outputs gives a bool variable. If a file in the path has been change -- true
    outputs:
      Ionos_dev: ${{ steps.filter.outputs.Ionos_dev }}
      Ionos_prod: ${{ steps.filter.outputs.Ionos_prod }}
      aws: ${{ steps.filter.outputs.aws }}
      azure: ${{ steps.filter.outputs.azure }}
    steps:
    #Use of an action which check if a file in a path has been change.
    - uses: dorny/paths-filter@v2
      id: filter
      with:
        filters: |
          Ionos_dev:
            - 'ionos/terraform/dev/**/*.tf'
          Ionos_prod:
            - 'ionos/terraform/prod/**/*.tf'
          aws:
            - 'aws/dns/domains/**/*.tf'
          azure:
            - 'azure/terraform/prod/**/*.tf'

  tfsec_scan_matrix:
    name: tfsec_sarif_report_all_directory
    runs-on: ubuntu-latest
    #Here we point the job changes, required for this job
    needs: changes
    #We create a matrix to store the output of each repo (true or false)
    #Each filter link with its directory (the directory is use to indicate the scan which directory it has to scan)
    strategy:
      matrix:
        include:
          - filters: ${{ needs.changes.outputs.Ionos_dev }}
            working_directory: ionos/terraform/dev/
          - filters: ${{ needs.changes.outputs.Ionos_prod }}
            working_directory: ionos/terraform/prod/
          - filters: ${{ needs.changes.outputs.aws }}
            working_directory: aws/dns/domains/
          - filters: ${{ needs.changes.outputs.azure }}
            working_directory: azure/terraform/prod/
    steps:
      #if the path has been modified, then clone repo, same thing for the others steps
      - if: ${{ matrix.filters == 'true' }}
        name: Clone repo
        uses: actions/checkout@master

      - if: ${{ matrix.filters == 'true' }}
        name: tfsec sarif ionos_dev
        uses: aquasecurity/[email protected]
        with:
          working_directory: ${{ matrix.working_directory }}
          sarif_file: tfsec.sarif

      - if: ${{ matrix.filters == 'true' }}
        name: Upload SARIF file
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: tfsec.sarif

What does it do? If a tf file has been change on a pull-request in a particular path then it runs on this specific path a tfsec scan.

To solve my problem: I implemented a matrix inside a job:

strategy:
      matrix:
        include:
          - filters: ${{ needs.changes.outputs.Ionos_dev }}
            working_directory: ionos/terraform/dev/
          - filters: ${{ needs.changes.outputs.Ionos_prod }}
            working_directory: ionos/terraform/prod/
          - filters: ${{ needs.changes.outputs.aws }}
            working_directory: aws/dns/domains/
          - filters: ${{ needs.changes.outputs.azure }}
            working_directory: azure/terraform/prod/

EXTRA: The "include" parameter, in my case, is to assigned an outputs to its specific path. However, if I wanted to combine all the possibilities, I would have done this way:

strategy:
      matrix:
        filter: [Ionos_dev, Ionos_prod, aws, azure]
        working_directory: [Ionos_dev, ionos/terraform/prod/, aws/dns/domains/, azure/terraform/prod/]

In this case, it will run all the 9 possibilities.

steps:
      #if the path has been modified, then clone repo, same thing for the others steps
      - if: ${{ matrix.filters == 'true' }}
        name: Clone repo
        uses: actions/checkout@master

      - if: ${{ matrix.filters == 'true' }}
        name: tfsec sarif ionos_dev
        uses: aquasecurity/[email protected]
        with:
          working_directory: ${{ matrix.working_directory }}
          sarif_file: tfsec.sarif

      - if: ${{ matrix.filters == 'true' }}
        name: Upload SARIF file
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: tfsec.sarif

For this part, I am still working on it. I try to improve it by simplify to only one 'if'