flink1.10集成PrometheusPushGateway,日志报错Failed to push metrics to PushGateway with jobName

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

flink1.10集成PrometheusPushGateway,日志报错Failed to push metrics to PushGateway with jobName

Jim Chen
Hi:
    我的环境是flink1.10.1,是基于yarn的per
job模式运行的。现在集成了PrometheusPushGateway,但是日志中,一直提示Failed to push metrics to
PushGateway with jobName。
    具体报错日志为:
2020-09-02 15:11:21.901 application_1598509186865_0129 172.22.64.72
wx11-dsj-flink004 [Flink-MetricRegistry-thread-1] WARN o.a.f.m.p.
PrometheusPushGatewayReporter - Failed to push metrics to PushGateway with
jobName realtime66b27e0fdfaa3860997abeb0170d84bb, groupingKey {}.
java.io.IOException: Response code from
http://172.16.24.146:9091/metrics/job/realtime66b27e0fdfaa3860997abeb0170d84bb
was 200
    at org.apache.flink.shaded.io.prometheus.client.exporter.PushGateway
.doRequest(PushGateway.java:297)
    at org.apache.flink.shaded.io.prometheus.client.exporter.PushGateway
.push(PushGateway.java:127)
    at org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter
.report(PrometheusPushGatewayReporter.java:109)
    at org.apache.flink.runtime.metrics.MetricRegistryImpl$ReporterTask.run(
MetricRegistryImpl.java:441)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:624)
    at java.lang.Thread.run(Thread.java:748)
Reply | Threaded
Open this post in threaded view
|

Re: flink1.10集成PrometheusPushGateway,日志报错Failed to push metrics to PushGateway with jobName

wanglei2@geekplus.com
你 pushgateway 是什么版本?版本降到 0.8.0 试一下




[hidden email]

 
Sender: Jim Chen
Send Time: 2020-09-02 17:43
Receiver: flink_user_zh
Subject: flink1.10集成PrometheusPushGateway,日志报错Failed to push metrics to PushGateway with jobName
Hi:
    我的环境是flink1.10.1,是基于yarn的per
job模式运行的。现在集成了PrometheusPushGateway,但是日志中,一直提示Failed to push metrics to
PushGateway with jobName。
    具体报错日志为:
2020-09-02 15:11:21.901 application_1598509186865_0129 172.22.64.72
wx11-dsj-flink004 [Flink-MetricRegistry-thread-1] WARN o.a.f.m.p.
PrometheusPushGatewayReporter - Failed to push metrics to PushGateway with
jobName realtime66b27e0fdfaa3860997abeb0170d84bb, groupingKey {}.
java.io.IOException: Response code from
http://172.16.24.146:9091/metrics/job/realtime66b27e0fdfaa3860997abeb0170d84bb
was 200
    at org.apache.flink.shaded.io.prometheus.client.exporter.PushGateway
.doRequest(PushGateway.java:297)
    at org.apache.flink.shaded.io.prometheus.client.exporter.PushGateway
.push(PushGateway.java:127)
    at org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter
.report(PrometheusPushGatewayReporter.java:109)
    at org.apache.flink.runtime.metrics.MetricRegistryImpl$ReporterTask.run(
MetricRegistryImpl.java:441)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:624)
    at java.lang.Thread.run(Thread.java:748)