Can we keep job section into a seperate file and is it possible to include prometheus.yml

Question

Can we keep the job section into a separate file and is it possible to include it in prometheus.yml.

We have multiple infrastructures to monitor,r so it would be helpful to aggregate with jobs and keep them in separate files. Pls help

Answer 1

You can keep each job in a different file and use the "file_sd_configs" clause to "include" them in the Prometheus config file, like in the following example.

prometheus.yaml

  - job_name: blackbox
    metrics_path: /probe
    file_sd_configs:
      - files:
          - blackbox.yaml

  - job_name: node
    scheme: http
    metrics_path: /metrics
    file_sd_configs:
      - files:
          - node.yaml

blackbox.yaml

- labels:
    module: icmp
  targets:
    - SERVER1
    - SERVER2

- labels:
    module: https_2xx
  targets:
    - https://SYSTEM1
    - https://SYSTEM2

node.yaml

- labels:
    group: prod
  targets:
    - SERVER1:9100
    - SERVER2:9100

- labels:
    group: dev
  targets:
    - SERVER3:9100
    - SERVER4:9100

Besides the organization, another advantage is that it is not necessary to reload the Prometheus settings every time these files are changed (this is done automatically).

More info in the Prometheus documentation here.