'How to run Prometheus monitoring with 1GB of RAM?

Having tried all kinds of flags to avoid keeping much data on the RAM has not succeeded. Even if I restrict "storage.local.memory-chunks" and also "chunks to persistence". I have played around with different values of both parameters, I have almost 1TB of secondary storage attached to it, but still I run out of RAM.

monitoringramprometheus


Solution 1:[1]

The settings on https://github.com/RobustPerception/demo_prometheus_ansible/blob/master/roles/prometheus/tasks/main.yml run with all the main components under 1GB total. Depending on how many active time series there are and how much data queries are touching you may need more RAM.

Solution 2:[2]

Prometheus memory usage depends on the following aspects:

  • The scrape_interval from Prometheus configs. Prometheus stores recently collected samples in memory for up to 2 hours. Bigger scrape_interval reduces the number of samples stored in memory. This reduces memory usage.
  • The number of active time series. Every active time series needs some memory. Bigger number of active time series needs more memory.
  • Types of performed queries. Prometheus loads raw samples for every query in memory. So heavy queries, which hit big number of time series and raw samples, may need more memory.

See more details about Prometheus memory usage in this and this article.

p.s. There are other Prometheus-like systems, which may need lower amounts of memory when dealing with the same workload. See, for example, VictoriaMetrics.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 brian-brazil
Solution 2 valyala

Related Questions