flink 1.12 通过rest api 提交任务时 SupervisorActor block

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

flink 1.12 通过rest api 提交任务时 SupervisorActor block

seuzxc
各位好:
     我这边flink 1.12运行一段时间后出现通过rest api提交任务 进行jobMaster
对应actor创建时被block,请问这种情况一般是什么原因呢?线程block信息如下(看源码这个位置只是向SupervisorActor发送消息注册一个jobmaster对应的actor,理论上不会有什么消耗资源的,但是我这线上环境有时会block一个小时然后恢复正常)

cluster-io-thread-14 BLOCKED blocked on java.lang.Object@67bb821e owned by
"cluster-io-thread-16" Id=218
at org.apache.flink.runtime.rpc.akka.AkkaRpcService.registerAkkaRpcActor
(AkkaRpcService.java:308)
at org.apache.flink.runtime.rpc.akka.AkkaRpcService.startServer
(AkkaRpcService.java:227)
at org.apache.flink.runtime.rpc.RpcEndpoint.<init> (RpcEndpoint.java:127)
at org.apache.flink.runtime.rpc.FencedRpcEndpoint.<init>
(FencedRpcEndpoint.java:48)
at org.apache.flink.runtime.jobmaster.JobMaster.<init> (JobMaster.java:235)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService
(DefaultJobMasterServiceFactory.java:95)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService
(DefaultJobMasterServiceFactory.java:39)
at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>
(JobManagerRunnerImpl.java:162)
at
org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner
(DefaultJobManagerRunnerFactory.java:86)
at
org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5
(Dispatcher.java:478)
at org.apache.flink.runtime.dispatcher.Dispatcher$$Lambda$796/1707020487.get
(flink-akka.actor.default-dispatcher-4684)
at java.util.concurrent.CompletableFuture$AsyncSupply.run
(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:624)
at java.lang.Thread.run


cluster-io-thread-16 WAITING waiting on
java.util.concurrent.CompletableFuture$Signaller@60dd3bae
at sun.misc.Unsafe.park (Native Method)
at java.util.concurrent.locks.LockSupport.park (LockSupport.java:175)
at java.util.concurrent.CompletableFuture$Signaller.block
(CompletableFuture.java:1707)
at java.util.concurrent.ForkJoinPool.managedBlock (ForkJoinPool.java:3323)
at java.util.concurrent.CompletableFuture.waitingGet
(CompletableFuture.java:1742)
at java.util.concurrent.CompletableFuture.join (CompletableFuture.java:1947)
at org.apache.flink.runtime.rpc.akka.SupervisorActor.startAkkaRpcActor
(SupervisorActor.java:208)
at org.apache.flink.runtime.rpc.akka.AkkaRpcService.registerAkkaRpcActor
(AkkaRpcService.java:311)
at org.apache.flink.runtime.rpc.akka.AkkaRpcService.startServer
(AkkaRpcService.java:227)
at org.apache.flink.runtime.rpc.RpcEndpoint.<init> (RpcEndpoint.java:127)
at org.apache.flink.runtime.rpc.FencedRpcEndpoint.<init>
(FencedRpcEndpoint.java:48)
at org.apache.flink.runtime.jobmaster.JobMaster.<init> (JobMaster.java:235)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService
(DefaultJobMasterServiceFactory.java:95)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService
(DefaultJobMasterServiceFactory.java:39)
at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>
(JobManagerRunnerImpl.java:162)
at
org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner
(DefaultJobManagerRunnerFactory.java:86)
at
org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5
(Dispatcher.java:478)
at org.apache.flink.runtime.dispatcher.Dispatcher$$Lambda$796/1707020487.get
(flink-akka.actor.default-dispatcher-4684)
at java.util.concurrent.CompletableFuture$AsyncSupply.run
(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker
(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:624)
at java.lang.Thread.run













--
Sent from: http://apache-flink.147419.n8.nabble.com/