'Flink taskmanager metrics disappear after starting a job
I have a Standalone cluster (with jobmanager and taskmanager on same machine) on 1.14.4 and I'm testing the migration to 1.15.0
But I keep losing the taskmanager metrics when I start a job on the 1.15 cluster
I use the same configuration as in the previous cluster
metrics.reporters: prom
metrics.reporter.prom.factory.class: org.apache.flink.metrics.prometheus.PrometheusReporterFactory
metrics.reporter.prom.port: 9250-9251
If the cluster is running without jobs I can see the metrics on port 9250 for jobmanager and on port 9251 for taskmanager
If I start a job, the metrics from taskmanager disappear and if I stop the job the metrics come live again
What am I missing?
Response
Its a bug on metrics of kafka source and sink on 1.15.0 https://lists.apache.org/thread/6bd9vmcroh7576d7h1kdcd8czf0b4l73
Is necessary to disable the registration of metrics on both
For source:
kafkaSourceBuilder.setProperty("register.consumer.metrics", "false");
For sink:
var producerProperties = new Properties();
producerProperties.setProperty("register.producer.metrics", "false");
producerSinkBuilder.setKafkaProducerConfig(producerProperties);
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|