flink yarn-perjob提交任务无法启动

[刘朋强]

问题：

通过如下命令提交任务到yarn-cluster，

flink run -m yarn-cluster -yjm 1024m -ytm 2048m -c org.apache.flink.streaming.examples.wordcount.WordCount /home/lpq/flink-examples-streaming_2.11.jar

在flink ui界面taskmanager总是0，任务无法启动，没有报错信息，不知道如何排查

yarn UI

flink ui

yarn container log

down cluster because application is in SUCCEEDED, diagnostics null.
2021-03-15 21:55:47,330 INFO  org.apache.flink.yarn.YarnResourceManagerDriver              [] - Unregister application from the YARN Resource Manager with final status SUCCEEDED.
2021-03-15 21:55:47,344 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl        [] - Waiting for application to be successfully unregistered.
2021-03-15 21:55:48,035 INFO  org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent [] - Closing components.
2021-03-15 21:55:48,036 INFO  org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcess [] - Stopping JobDispatcherLeaderProcess.
2021-03-15 21:55:48,037 INFO  org.apache.flink.runtime.dispatcher.MiniDispatcher           [] - Stopping dispatcher akka.tcp://flink@bdp:38344/user/rpc/dispatcher_1.
2021-03-15 21:55:48,037 INFO  org.apache.flink.runtime.dispatcher.MiniDispatcher           [] - Stopping all currently running jobs of dispatcher akka.tcp://flink@bdp:38344/user/rpc/dispatcher_1.
2021-03-15 21:55:48,037 INFO  org.apache.flink.runtime.rest.handler.legacy.backpressure.BackPressureRequestCoordinator [] - Shutting down back pressure request coordinator.
2021-03-15 21:55:48,037 INFO  org.apache.flink.runtime.dispatcher.MiniDispatcher           [] - Stopped dispatcher akka.tcp://flink@bdp:38344/user/rpc/dispatcher_1.
2021-03-15 21:55:48,040 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] - Opening proxy : bdp:33576
2021-03-15 21:55:48,041 INFO  org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl [] - Interrupted while waiting for queue
java.lang.InterruptedException: null
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_121]
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_121]
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) ~[?:1.8.0_121]
	at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:287) [hadoop-yarn-client-2.7.7.jar:?]
2021-03-15 21:55:48,066 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] - Closing the SlotManager.
2021-03-15 21:55:48,066 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] - Suspending the SlotManager.
2021-03-15 21:55:48,069 INFO  org.apache.flink.runtime.blob.BlobServer                     [] - Stopped BLOB server at 0.0.0.0:34775
2021-03-15 21:55:48,069 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopping Akka RPC service.
2021-03-15 21:55:48,076 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopping Akka RPC service.
2021-03-15 21:55:48,093 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Shutting down remote daemon.
2021-03-15 21:55:48,093 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Shutting down remote daemon.
2021-03-15 21:55:48,093 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Remote daemon shut down; proceeding with flushing remote transports.
2021-03-15 21:55:48,095 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Remote daemon shut down; proceeding with flushing remote transports.
2021-03-15 21:55:48,111 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Remoting shut down.
2021-03-15 21:55:48,119 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Remoting shut down.
2021-03-15 21:55:48,129 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopped Akka RPC service.
2021-03-15 21:55:48,130 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopped Akka RPC service.
2021-03-15 21:55:48,132 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Terminating cluster entrypoint process YarnJobClusterEntrypoint with exit code 0.

[lian]

两种情况：
情况1：jar打包不完整，重新打包试一下
情况2：缺少依赖


在2021年03月15日 21:59，刘朋强 写道：
问题：
通过如下命令提交任务到yarn-cluster，
flink run -m yarn-cluster -yjm 1024m -ytm 2048m -c org.apache.flink.streaming.examples.wordcount.WordCount /home/lpq/flink-examples-streaming_2.11.jar
在flink ui界面taskmanager总是0，任务无法启动，没有报错信息，不知道如何排查
yarn UI
flink ui


yarn container log
down cluster because application is in SUCCEEDED, diagnostics null.
2021-03-15 21:55:47,330 INFO  org.apache.flink.yarn.YarnResourceManagerDriver              [] - Unregister application from the YARN Resource Manager with final status SUCCEEDED.
2021-03-15 21:55:47,344 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl        [] - Waiting for application to be successfully unregistered.
2021-03-15 21:55:48,035 INFO  org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent [] - Closing components.
2021-03-15 21:55:48,036 INFO  org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcess [] - Stopping JobDispatcherLeaderProcess.
2021-03-15 21:55:48,037 INFO  org.apache.flink.runtime.dispatcher.MiniDispatcher           [] - Stopping dispatcher akka.tcp://flink@bdp:38344/user/rpc/dispatcher_1.
2021-03-15 21:55:48,037 INFO  org.apache.flink.runtime.dispatcher.MiniDispatcher           [] - Stopping all currently running jobs of dispatcher akka.tcp://flink@bdp:38344/user/rpc/dispatcher_1.
2021-03-15 21:55:48,037 INFO  org.apache.flink.runtime.rest.handler.legacy.backpressure.BackPressureRequestCoordinator [] - Shutting down back pressure request coordinator.
2021-03-15 21:55:48,037 INFO  org.apache.flink.runtime.dispatcher.MiniDispatcher           [] - Stopped dispatcher akka.tcp://flink@bdp:38344/user/rpc/dispatcher_1.
2021-03-15 21:55:48,040 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] - Opening proxy : bdp:33576
2021-03-15 21:55:48,041 INFO  org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl [] - Interrupted while waiting for queue
java.lang.InterruptedException: null
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_121]
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_121]
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) ~[?:1.8.0_121]
        at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:287) [hadoop-yarn-client-2.7.7.jar:?]
2021-03-15 21:55:48,066 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] - Closing the SlotManager.
2021-03-15 21:55:48,066 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] - Suspending the SlotManager.
2021-03-15 21:55:48,069 INFO  org.apache.flink.runtime.blob.BlobServer                     [] - Stopped BLOB server at 0.0.0.0:34775
2021-03-15 21:55:48,069 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopping Akka RPC service.
2021-03-15 21:55:48,076 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopping Akka RPC service.
2021-03-15 21:55:48,093 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Shutting down remote daemon.
2021-03-15 21:55:48,093 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Shutting down remote daemon.
2021-03-15 21:55:48,093 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Remote daemon shut down; proceeding with flushing remote transports.
2021-03-15 21:55:48,095 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Remote daemon shut down; proceeding with flushing remote transports.
2021-03-15 21:55:48,111 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Remoting shut down.
2021-03-15 21:55:48,119 INFO  akka.remote.RemoteActorRefProvider$RemotingTerminator        [] - Remoting shut down.
2021-03-15 21:55:48,129 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopped Akka RPC service.
2021-03-15 21:55:48,130 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopped Akka RPC service.
2021-03-15 21:55:48,132 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Terminating cluster entrypoint process YarnJobClusterEntrypoint with exit code 0.