Apache Flink 中文用户邮件列表

flink on yarn 指定节点开启 yarn session 报错

Classic

List

Threaded

8 messages Options

杨浩程

flink on yarn 指定节点开启 yarn session 报错

各位大佬好！

请教各位个问题：使用的flink 版本1.9.0。测试flink on yarn 指定节点开启yarn session 会话。

希望yarnsession开启的集群 jobmanager开启在我指定的节点上。

更改的配置如下：

报错信息如下：

===============================================================================================================

2019-11-27 10:35:45,640 INFO org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperati

2019-11-27 10:35:45,650 INFO org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Session: 0x36ea133e2b51435 closed

2019-11-27 10:35:45,650 INFO org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x3

2019-11-27 10:35:45,651 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping Akka RPC service.

2019-11-27 10:35:45,659 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping Akka RPC service.

2019-11-27 10:35:45,666 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon.

2019-11-27 10:35:45,669 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with fl

2019-11-27 10:35:45,676 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon.

2019-11-27 10:35:45,677 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with fl

2019-11-27 10:35:45,696 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting shut down.

2019-11-27 10:35:45,709 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped Akka RPC service.

2019-11-27 10:35:45,713 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped Akka RPC service.

2019-11-27 10:35:45,714 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could not start cluster entrypoint YarnSess

org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint YarnSessionClusterEntrypoint.

at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182)

at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)

at org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)

Caused by: org.apache.flink.util.FlinkException: Could not create the DispatcherResourceManagerComponent.

at org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou

at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)

at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)

at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)

at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)

... 2 more

Caused by: java.net.BindException: Could not start rest endpoint on any port in port range 8082

at org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)

at org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou

... 9 more

End of LogType:jobmanager.log

===============================================================================================================

请问这种问题要怎么解决？多谢各位大佬！

tison

Re: flink on yarn 指定节点开启 yarn session 报错

8082 端口被占用了吧？

Best,
tison.

杨浩程 <[hidden email]> 于2019年11月27日周三上午10:56写道：

> 各位大佬好！
> 请教各位个问题：使用的flink 版本1.9.0。测试flink on yarn 指定节点开启yarn session 会话。
> 希望yarnsession开启的集群 jobmanager开启在我指定的节点上。
> 更改的配置如下：
> 报错信息如下：
>
> ===============================================================================================================
> 2019-11-27 10:35:45,640 INFO
> org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
> - backgroundOperati
> 2019-11-27 10:35:45,650 INFO
> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper -
> Session: 0x36ea133e2b51435 closed
> 2019-11-27 10:35:45,650 INFO
> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
> EventThread shut down for session: 0x3
> 2019-11-27 10:35:45,651 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping
> Akka RPC service.
> 2019-11-27 10:35:45,659 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping
> Akka RPC service.
> 2019-11-27 10:35:45,666 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting
> down remote daemon.
> 2019-11-27 10:35:45,669 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
> daemon shut down; proceeding with fl
> 2019-11-27 10:35:45,676 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting
> down remote daemon.
> 2019-11-27 10:35:45,677 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
> daemon shut down; proceeding with fl
> 2019-11-27 10:35:45,696 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting
> shut down.
> 2019-11-27 10:35:45,696 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting
> shut down.
> 2019-11-27 10:35:45,709 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped
> Akka RPC service.
> 2019-11-27 10:35:45,713 INFO
> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped
> Akka RPC service.
> 2019-11-27 10:35:45,714 ERROR
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could not
> start cluster entrypoint YarnSess
> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to
> initialize the cluster entrypoint YarnSessionClusterEntrypoint.
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)
> at
> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)
> Caused by: org.apache.flink.util.FlinkException: Could not create the
> DispatcherResourceManagerComponent.
> at
> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)
> ... 2 more
> Caused by: java.net.BindException: Could not start rest endpoint on any
> port in port range 8082
> at
> org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)
> at
> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
> ... 9 more
>
> End of LogType:jobmanager.log
>
> ===============================================================================================================
> 请问这种问题要怎么解决？多谢各位大佬！
>
>
>
>

Yangze Guo

Re: flink on yarn 指定节点开启 yarn session 报错

In reply to this post by 杨浩程

您好，目前ML不支持图片，能将配置贴上来么？
另外看报错信息像是8082端口被占用导致rest服务起不起来

Best,
Yangze Guo

On Wed, Nov 27, 2019 at 10:56 AM 杨浩程 <[hidden email]> wrote:

杨浩程

Re:Re: flink on yarn 指定节点开启 yarn session 报错

In reply to this post by tison

tison 你好，
我netstat查看了想要指定的节点的端口情况，8082没有被占用。所以有点困惑这个报错。

在 2019-11-27 10:59:52，"tison" <[hidden email]> 写道：

>8082 端口被占用了吧？
>
>Best,
>tison.
>
>
>杨浩程 <[hidden email]> 于2019年11月27日周三上午10:56写道：
>
>> 各位大佬好！
>> 请教各位个问题：使用的flink 版本1.9.0。测试flink on yarn 指定节点开启yarn session 会话。
>> 希望yarnsession开启的集群 jobmanager开启在我指定的节点上。
>> 更改的配置如下：
>> 报错信息如下：
>>
>> ===============================================================================================================
>> 2019-11-27 10:35:45,640 INFO
>> org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
>> - backgroundOperati
>> 2019-11-27 10:35:45,650 INFO
>> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper -
>> Session: 0x36ea133e2b51435 closed
>> 2019-11-27 10:35:45,650 INFO
>> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
>> EventThread shut down for session: 0x3
>> 2019-11-27 10:35:45,651 INFO
>> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping
>> Akka RPC service.
>> 2019-11-27 10:35:45,659 INFO
>> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping
>> Akka RPC service.
>> 2019-11-27 10:35:45,666 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting
>> down remote daemon.
>> 2019-11-27 10:35:45,669 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
>> daemon shut down; proceeding with fl
>> 2019-11-27 10:35:45,676 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting
>> down remote daemon.
>> 2019-11-27 10:35:45,677 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
>> daemon shut down; proceeding with fl
>> 2019-11-27 10:35:45,696 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting
>> shut down.
>> 2019-11-27 10:35:45,696 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting
>> shut down.
>> 2019-11-27 10:35:45,709 INFO
>> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped
>> Akka RPC service.
>> 2019-11-27 10:35:45,713 INFO
>> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped
>> Akka RPC service.
>> 2019-11-27 10:35:45,714 ERROR
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could not
>> start cluster entrypoint YarnSess
>> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to
>> initialize the cluster entrypoint YarnSessionClusterEntrypoint.
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182)
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)
>> at
>> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)
>> Caused by: org.apache.flink.util.FlinkException: Could not create the
>> DispatcherResourceManagerComponent.
>> at
>> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:422)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
>> at
>> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)
>> ... 2 more
>> Caused by: java.net.BindException: Could not start rest endpoint on any
>> port in port range 8082
>> at
>> org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)
>> at
>> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
>> ... 9 more
>>
>> End of LogType:jobmanager.log
>>
>> ===============================================================================================================
>> 请问这种问题要怎么解决？多谢各位大佬！
>>
>>
>>
>>

杨浩程

Re:Re: flink on yarn 指定节点开启 yarn session 报错

In reply to this post by Yangze Guo

================================================================================
好的，更改过的配置如下：

#==============================================================================
# Rest & web frontend
#==============================================================================

# The port to which the REST client connects to. If rest.bind-port has
# not been specified, then the server will bind to this port as well.
#
rest.port: 8082

# The address to which the REST client will connect to
#
rest.address: bigdata-test-8

# Port range for the REST and web server to bind to.
#
#rest.bind-port: 8080-8090
rest.bind-port: 8082

# The address that the REST & web server binds to
#
rest.bind-address: bigdata-test-8

# Flag to specify whether job submission is enabled from the web-based
# runtime monitor. Uncomment to disable.

#web.submit.enable: false
#web.upload.dir: /data/flink-1.9.0/upload_jars/
================================================================================

在 2019-11-27 11:01:47，"Yangze Guo" <[hidden email]> 写道：

>您好，目前ML不支持图片，能将配置贴上来么？
>另外看报错信息像是8082端口被占用导致rest服务起不起来
>
>Best,
>Yangze Guo
>
>
>On Wed, Nov 27, 2019 at 10:56 AM 杨浩程 <[hidden email]> wrote:
>
>> 各位大佬好！
>> 请教各位个问题：使用的flink 版本1.9.0。测试flink on yarn 指定节点开启yarn session 会话。
>> 希望yarnsession开启的集群 jobmanager开启在我指定的节点上。
>> 更改的配置如下：
>> 报错信息如下：
>>
>> ===============================================================================================================
>> 2019-11-27 10:35:45,640 INFO
>> org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
>> - backgroundOperati
>> 2019-11-27 10:35:45,650 INFO
>> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper -
>> Session: 0x36ea133e2b51435 closed
>> 2019-11-27 10:35:45,650 INFO
>> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
>> EventThread shut down for session: 0x3
>> 2019-11-27 10:35:45,651 INFO
>> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping
>> Akka RPC service.
>> 2019-11-27 10:35:45,659 INFO
>> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping
>> Akka RPC service.
>> 2019-11-27 10:35:45,666 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting
>> down remote daemon.
>> 2019-11-27 10:35:45,669 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
>> daemon shut down; proceeding with fl
>> 2019-11-27 10:35:45,676 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting
>> down remote daemon.
>> 2019-11-27 10:35:45,677 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
>> daemon shut down; proceeding with fl
>> 2019-11-27 10:35:45,696 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting
>> shut down.
>> 2019-11-27 10:35:45,696 INFO
>> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting
>> shut down.
>> 2019-11-27 10:35:45,709 INFO
>> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped
>> Akka RPC service.
>> 2019-11-27 10:35:45,713 INFO
>> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped
>> Akka RPC service.
>> 2019-11-27 10:35:45,714 ERROR
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could not
>> start cluster entrypoint YarnSess
>> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to
>> initialize the cluster entrypoint YarnSessionClusterEntrypoint.
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182)
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)
>> at
>> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)
>> Caused by: org.apache.flink.util.FlinkException: Could not create the
>> DispatcherResourceManagerComponent.
>> at
>> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:422)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
>> at
>> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>> at
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)
>> ... 2 more
>> Caused by: java.net.BindException: Could not start rest endpoint on any
>> port in port range 8082
>> at
>> org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)
>> at
>> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
>> ... 9 more
>>
>> End of LogType:jobmanager.log
>>
>> ===============================================================================================================
>> 请问这种问题要怎么解决？多谢各位大佬！
>>
>>
>>
>>

Yangze Guo

Re: Re: flink on yarn 指定节点开启 yarn session 报错

8082我记得是historyserver.web.port的默认值，很可能是冲突了改成8081呢？

Best,
Yangze Guo

On Wed, Nov 27, 2019 at 11:13 AM 杨浩程 <[hidden email]> wrote:

>
> ================================================================================
> 好的，更改过的配置如下：
>
> #==============================================================================
> # Rest & web frontend
> #==============================================================================
>
>
> # The port to which the REST client connects to. If rest.bind-port has
> # not been specified, then the server will bind to this port as well.
> #
> rest.port: 8082
>
>
> # The address to which the REST client will connect to
> #
> rest.address: bigdata-test-8
>
>
> # Port range for the REST and web server to bind to.
> #
> #rest.bind-port: 8080-8090
> rest.bind-port: 8082
>
>
> # The address that the REST & web server binds to
> #
> rest.bind-address: bigdata-test-8
>
>
> # Flag to specify whether job submission is enabled from the web-based
> # runtime monitor. Uncomment to disable.
>
>
> #web.submit.enable: false
> #web.upload.dir: /data/flink-1.9.0/upload_jars/
> ================================================================================
>
>
>
>
>
> 在 2019-11-27 11:01:47，"Yangze Guo" <[hidden email]> 写道：
> >您好，目前ML不支持图片，能将配置贴上来么？
> >另外看报错信息像是8082端口被占用导致rest服务起不起来
> >
> >Best,
> >Yangze Guo
> >
> >
> >On Wed, Nov 27, 2019 at 10:56 AM 杨浩程 <[hidden email]> wrote:
> >
> >> 各位大佬好！
> >> 请教各位个问题：使用的flink 版本1.9.0。测试flink on yarn 指定节点开启yarn session 会话。
> >> 希望yarnsession开启的集群 jobmanager开启在我指定的节点上。
> >> 更改的配置如下：
> >> 报错信息如下：
> >>
> >> ===============================================================================================================
> >> 2019-11-27 10:35:45,640 INFO
> >> org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
> >> - backgroundOperati
> >> 2019-11-27 10:35:45,650 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper -
> >> Session: 0x36ea133e2b51435 closed
> >> 2019-11-27 10:35:45,650 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
> >> EventThread shut down for session: 0x3
> >> 2019-11-27 10:35:45,651 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping
> >> Akka RPC service.
> >> 2019-11-27 10:35:45,659 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopping
> >> Akka RPC service.
> >> 2019-11-27 10:35:45,666 INFO
> >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting
> >> down remote daemon.
> >> 2019-11-27 10:35:45,669 INFO
> >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
> >> daemon shut down; proceeding with fl
> >> 2019-11-27 10:35:45,676 INFO
> >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting
> >> down remote daemon.
> >> 2019-11-27 10:35:45,677 INFO
> >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
> >> daemon shut down; proceeding with fl
> >> 2019-11-27 10:35:45,696 INFO
> >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting
> >> shut down.
> >> 2019-11-27 10:35:45,696 INFO
> >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting
> >> shut down.
> >> 2019-11-27 10:35:45,709 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped
> >> Akka RPC service.
> >> 2019-11-27 10:35:45,713 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcService - Stopped
> >> Akka RPC service.
> >> 2019-11-27 10:35:45,714 ERROR
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could not
> >> start cluster entrypoint YarnSess
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to
> >> initialize the cluster entrypoint YarnSessionClusterEntrypoint.
> >> at
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182)
> >> at
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)
> >> at
> >> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)
> >> Caused by: org.apache.flink.util.FlinkException: Could not create the
> >> DispatcherResourceManagerComponent.
> >> at
> >> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
> >> at
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)
> >> at
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)
> >> at java.security.AccessController.doPrivileged(Native Method)
> >> at javax.security.auth.Subject.doAs(Subject.java:422)
> >> at
> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> >> at
> >> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> >> at
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)
> >> ... 2 more
> >> Caused by: java.net.BindException: Could not start rest endpoint on any
> >> port in port range 8082
> >> at
> >> org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)
> >> at
> >> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
> >> ... 9 more
> >>
> >> End of LogType:jobmanager.log
> >>
> >> ===============================================================================================================
> >> 请问这种问题要怎么解决？多谢各位大佬！
> >>
> >>
> >>
> >>

tison

Re: Re: flink on yarn 指定节点开启 yarn session 报错

确实是 historyserver.web.port 的默认值，但我记得 HistoryServer 默认是不启动的

总之可以确认下问题能否稳定复现，失败的瞬间端口有无被占用，以及换个端口能不能起来

如果还不行再找找其他原因

Best,
tison.

Yangze Guo <[hidden email]> 于2019年11月27日周三上午11:23写道：

> 8082我记得是historyserver.web.port的默认值，很可能是冲突了改成8081呢？
>
> Best,
> Yangze Guo
>
> On Wed, Nov 27, 2019 at 11:13 AM 杨浩程 <[hidden email]> wrote:
> >
> >
> ================================================================================
> > 好的，更改过的配置如下：
> >
> >
> #==============================================================================
> > # Rest & web frontend
> >
> #==============================================================================
> >
> >
> > # The port to which the REST client connects to. If rest.bind-port has
> > # not been specified, then the server will bind to this port as well.
> > #
> > rest.port: 8082
> >
> >
> > # The address to which the REST client will connect to
> > #
> > rest.address: bigdata-test-8
> >
> >
> > # Port range for the REST and web server to bind to.
> > #
> > #rest.bind-port: 8080-8090
> > rest.bind-port: 8082
> >
> >
> > # The address that the REST & web server binds to
> > #
> > rest.bind-address: bigdata-test-8
> >
> >
> > # Flag to specify whether job submission is enabled from the web-based
> > # runtime monitor. Uncomment to disable.
> >
> >
> > #web.submit.enable: false
> > #web.upload.dir: /data/flink-1.9.0/upload_jars/
> >
> ================================================================================
> >
> >
> >
> >
> >
> > 在 2019-11-27 11:01:47，"Yangze Guo" <[hidden email]> 写道：
> > >您好，目前ML不支持图片，能将配置贴上来么？
> > >另外看报错信息像是8082端口被占用导致rest服务起不起来
> > >
> > >Best,
> > >Yangze Guo
> > >
> > >
> > >On Wed, Nov 27, 2019 at 10:56 AM 杨浩程 <[hidden email]> wrote:
> > >
> > >> 各位大佬好！
> > >> 请教各位个问题：使用的flink 版本1.9.0。测试flink on yarn 指定节点开启yarn session
> 会话。
> > >> 希望yarnsession开启的集群 jobmanager开启在我指定的节点上。
> > >> 更改的配置如下：
> > >> 报错信息如下：
> > >>
> > >>
> ===============================================================================================================
> > >> 2019-11-27 10:35:45,640 INFO
> > >>
> org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
> > >> - backgroundOperati
> > >> 2019-11-27 10:35:45,650 INFO
> > >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper -
> > >> Session: 0x36ea133e2b51435 closed
> > >> 2019-11-27 10:35:45,650 INFO
> > >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
> > >> EventThread shut down for session: 0x3
> > >> 2019-11-27 10:35:45,651 INFO
> > >> org.apache.flink.runtime.rpc.akka.AkkaRpcService -
> Stopping
> > >> Akka RPC service.
> > >> 2019-11-27 10:35:45,659 INFO
> > >> org.apache.flink.runtime.rpc.akka.AkkaRpcService -
> Stopping
> > >> Akka RPC service.
> > >> 2019-11-27 10:35:45,666 INFO
> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator -
> Shutting
> > >> down remote daemon.
> > >> 2019-11-27 10:35:45,669 INFO
> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
> > >> daemon shut down; proceeding with fl
> > >> 2019-11-27 10:35:45,676 INFO
> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator -
> Shutting
> > >> down remote daemon.
> > >> 2019-11-27 10:35:45,677 INFO
> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
> > >> daemon shut down; proceeding with fl
> > >> 2019-11-27 10:35:45,696 INFO
> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator -
> Remoting
> > >> shut down.
> > >> 2019-11-27 10:35:45,696 INFO
> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator -
> Remoting
> > >> shut down.
> > >> 2019-11-27 10:35:45,709 INFO
> > >> org.apache.flink.runtime.rpc.akka.AkkaRpcService -
> Stopped
> > >> Akka RPC service.
> > >> 2019-11-27 10:35:45,713 INFO
> > >> org.apache.flink.runtime.rpc.akka.AkkaRpcService -
> Stopped
> > >> Akka RPC service.
> > >> 2019-11-27 10:35:45,714 ERROR
> > >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could
> not
> > >> start cluster entrypoint YarnSess
> > >> org.apache.flink.runtime.entrypoint.ClusterEntrypointException:
> Failed to
> > >> initialize the cluster entrypoint YarnSessionClusterEntrypoint.
> > >> at
> > >>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182)
> > >> at
> > >>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)
> > >> at
> > >>
> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)
> > >> Caused by: org.apache.flink.util.FlinkException: Could not create the
> > >> DispatcherResourceManagerComponent.
> > >> at
> > >>
> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
> > >> at
> > >>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)
> > >> at
> > >>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)
> > >> at java.security.AccessController.doPrivileged(Native Method)
> > >> at javax.security.auth.Subject.doAs(Subject.java:422)
> > >> at
> > >>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
> > >> at
> > >>
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> > >> at
> > >>
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)
> > >> ... 2 more
> > >> Caused by: java.net.BindException: Could not start rest endpoint on
> any
> > >> port in port range 8082
> > >> at
> > >>
> org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)
> > >> at
> > >>
> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
> > >> ... 9 more
> > >>
> > >> End of LogType:jobmanager.log
> > >>
> > >>
> ===============================================================================================================
> > >> 请问这种问题要怎么解决？多谢各位大佬！
> > >>
> > >>
> > >>
> > >>
>

杨浩程

Re:Re: Re: flink on yarn 指定节点开启 yarn session 报错

hi，我确保，端口没占用的情况下更换rest.port: 8072，且更改了配置：

#==============================================================================
# HistoryServer
#==============================================================================

# The HistoryServer is started and stopped via bin/historyserver.sh (start|stop)

# Directory to upload completed jobs to. Add this directory to the list of
# monitored directories of the HistoryServer as well (see below).
jobmanager.archive.fs.dir: hdfs://bigdata-test-8:18082/completed-jobs/

# The address under which the web-based HistoryServer listens.
historyserver.web.address: bigdata-test-8

# The port under which the web-based HistoryServer listens.
historyserver.web.port: 18082

# Comma separated list of directories to monitor for completed jobs.
historyserver.archive.fs.dir: hdfs://bigdata-test-8:18082/completed-jobs/

# Interval in milliseconds for refreshing the monitored directories.
historyserver.archive.fs.refresh-interval: 10000
========================================================================
但在这种情况下启动yarn session 报错信息如下：
=========================================================================
2019-11-27 14:50:40,981 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could not start cluster entrypoint YarnSessionClusterEntrypoint.
org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint YarnSessionClusterEntrypoint.
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)
at org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)
Caused by: org.apache.flink.util.FlinkException: Could not create the DispatcherResourceManagerComponent.
at org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResourceManagerComponentFactory.java:257)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)
... 2 more
Caused by: java.net.BindException: Could not start rest endpoint on any port in port range 8070-8090
at org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)
at org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResourceManagerComponentFactory.java:161)
... 9 more
================================================================================

在 2019-11-27 11:28:06，"tison" <[hidden email]> 写道：

>确实是 historyserver.web.port 的默认值，但我记得 HistoryServer 默认是不启动的
>
>总之可以确认下问题能否稳定复现，失败的瞬间端口有无被占用，以及换个端口能不能起来
>
>如果还不行再找找其他原因
>
>Best,
>tison.
>
>
>Yangze Guo <[hidden email]> 于2019年11月27日周三上午11:23写道：
>
>> 8082我记得是historyserver.web.port的默认值，很可能是冲突了改成8081呢？
>>
>> Best,
>> Yangze Guo
>>
>> On Wed, Nov 27, 2019 at 11:13 AM 杨浩程 <[hidden email]> wrote:
>> >
>> >
>> ================================================================================
>> > 好的，更改过的配置如下：
>> >
>> >
>> #==============================================================================
>> > # Rest & web frontend
>> >
>> #==============================================================================
>> >
>> >
>> > # The port to which the REST client connects to. If rest.bind-port has
>> > # not been specified, then the server will bind to this port as well.
>> > #
>> > rest.port: 8082
>> >
>> >
>> > # The address to which the REST client will connect to
>> > #
>> > rest.address: bigdata-test-8
>> >
>> >
>> > # Port range for the REST and web server to bind to.
>> > #
>> > #rest.bind-port: 8080-8090
>> > rest.bind-port: 8082
>> >
>> >
>> > # The address that the REST & web server binds to
>> > #
>> > rest.bind-address: bigdata-test-8
>> >
>> >
>> > # Flag to specify whether job submission is enabled from the web-based
>> > # runtime monitor. Uncomment to disable.
>> >
>> >
>> > #web.submit.enable: false
>> > #web.upload.dir: /data/flink-1.9.0/upload_jars/
>> >
>> ================================================================================
>> >
>> >
>> >
>> >
>> >
>> > 在 2019-11-27 11:01:47，"Yangze Guo" <[hidden email]> 写道：
>> > >您好，目前ML不支持图片，能将配置贴上来么？
>> > >另外看报错信息像是8082端口被占用导致rest服务起不起来
>> > >
>> > >Best,
>> > >Yangze Guo
>> > >
>> > >
>> > >On Wed, Nov 27, 2019 at 10:56 AM 杨浩程 <[hidden email]> wrote:
>> > >
>> > >> 各位大佬好！
>> > >> 请教各位个问题：使用的flink 版本1.9.0。测试flink on yarn 指定节点开启yarn session
>> 会话。
>> > >> 希望yarnsession开启的集群 jobmanager开启在我指定的节点上。
>> > >> 更改的配置如下：
>> > >> 报错信息如下：
>> > >>
>> > >>
>> ===============================================================================================================
>> > >> 2019-11-27 10:35:45,640 INFO
>> > >>
>> org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
>> > >> - backgroundOperati
>> > >> 2019-11-27 10:35:45,650 INFO
>> > >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper -
>> > >> Session: 0x36ea133e2b51435 closed
>> > >> 2019-11-27 10:35:45,650 INFO
>> > >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
>> > >> EventThread shut down for session: 0x3
>> > >> 2019-11-27 10:35:45,651 INFO
>> > >> org.apache.flink.runtime.rpc.akka.AkkaRpcService -
>> Stopping
>> > >> Akka RPC service.
>> > >> 2019-11-27 10:35:45,659 INFO
>> > >> org.apache.flink.runtime.rpc.akka.AkkaRpcService -
>> Stopping
>> > >> Akka RPC service.
>> > >> 2019-11-27 10:35:45,666 INFO
>> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator -
>> Shutting
>> > >> down remote daemon.
>> > >> 2019-11-27 10:35:45,669 INFO
>> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
>> > >> daemon shut down; proceeding with fl
>> > >> 2019-11-27 10:35:45,676 INFO
>> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator -
>> Shutting
>> > >> down remote daemon.
>> > >> 2019-11-27 10:35:45,677 INFO
>> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote
>> > >> daemon shut down; proceeding with fl
>> > >> 2019-11-27 10:35:45,696 INFO
>> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator -
>> Remoting
>> > >> shut down.
>> > >> 2019-11-27 10:35:45,696 INFO
>> > >> akka.remote.RemoteActorRefProvider$RemotingTerminator -
>> Remoting
>> > >> shut down.
>> > >> 2019-11-27 10:35:45,709 INFO
>> > >> org.apache.flink.runtime.rpc.akka.AkkaRpcService -
>> Stopped
>> > >> Akka RPC service.
>> > >> 2019-11-27 10:35:45,713 INFO
>> > >> org.apache.flink.runtime.rpc.akka.AkkaRpcService -
>> Stopped
>> > >> Akka RPC service.
>> > >> 2019-11-27 10:35:45,714 ERROR
>> > >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Could
>> not
>> > >> start cluster entrypoint YarnSess
>> > >> org.apache.flink.runtime.entrypoint.ClusterEntrypointException:
>> Failed to
>> > >> initialize the cluster entrypoint YarnSessionClusterEntrypoint.
>> > >> at
>> > >>
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182)
>> > >> at
>> > >>
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)
>> > >> at
>> > >>
>> org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)
>> > >> Caused by: org.apache.flink.util.FlinkException: Could not create the
>> > >> DispatcherResourceManagerComponent.
>> > >> at
>> > >>
>> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
>> > >> at
>> > >>
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)
>> > >> at
>> > >>
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)
>> > >> at java.security.AccessController.doPrivileged(Native Method)
>> > >> at javax.security.auth.Subject.doAs(Subject.java:422)
>> > >> at
>> > >>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
>> > >> at
>> > >>
>> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>> > >> at
>> > >>
>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)
>> > >> ... 2 more
>> > >> Caused by: java.net.BindException: Could not start rest endpoint on
>> any
>> > >> port in port range 8082
>> > >> at
>> > >>
>> org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:219)
>> > >> at
>> > >>
>> org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResou
>> > >> ... 9 more
>> > >>
>> > >> End of LogType:jobmanager.log
>> > >>
>> > >>
>> ===============================================================================================================
>> > >> 请问这种问题要怎么解决？多谢各位大佬！
>> > >>
>> > >>
>> > >>
>> > >>
>>