flink1.10 yarn模式无法提交作业

[Fei Han]

@all:
   在Flink1.10中，用yarn模式无法提交作业。
  提示如下：
lease check if the requested resources are available in the YARN cluster
2020-02-09 17:22:26,318 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2020-02-09 17:22:26,570 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2020-02-09 17:22:26,822 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2020-02-09 17:22:27,074 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2020-02-09 17:22:27,326 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2020-02-09 17:22:27,578 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2020-02-09 17:22:27,831 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2020-02-09 17:22:28,083 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
2020-02-09 17:22:28,336 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster


另外，我看了yarn资源足够。但就是提交不了。
我的作业提交命令如下：
 ./bin/flink run -m yarn-cluster  -yjm 1024  -ytm 2048  /opt/flink-1.10/examples/batch/WordCount.jar  --input hdfs://192.168.xxx.xxx:9000/test/LICENSE  --output hdfs://192.168.xxx.xxx:9000/test/result.txt

[Jun Zhang]

你看看是不是yarn队列有很多accept状态的任务，如果有很多正在排队的任务，那就是可能超过了flink设置的超时时间还没有来得及给你这个任务分配资源，就会报这个错误。
但是这个不影响任务的提交，等一会任务提交成功了，客户端就会返回提交成功的结果。

Fei Han <[hidden email]> 于2020年2月9日周日 下午5:35写道：

> @all:
>    在Flink1.10中，用yarn模式无法提交作业。
>   提示如下：
> lease check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:26,318 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
> 2020-02-09 17:22:26,570 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
> 2020-02-09 17:22:26,822 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
> 2020-02-09 17:22:27,074 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
> 2020-02-09 17:22:27,326 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
> 2020-02-09 17:22:27,578 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
> 2020-02-09 17:22:27,831 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
> 2020-02-09 17:22:28,083 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
> 2020-02-09 17:22:28,336 INFO  org.apache.flink.yarn.YarnClusterDescriptor
>                  - Deployment took more than 60 seconds. Please check if
> the requested resources are available in the YARN cluster
>
>
> 另外，我看了yarn资源足够。但就是提交不了。
> 我的作业提交命令如下：
>  ./bin/flink run -m yarn-cluster  -yjm 1024  -ytm 2048
> /opt/flink-1.10/examples/batch/WordCount.jar  --input
> hdfs://192.168.xxx.xxx:9000/test/LICENSE  --output
> hdfs://192.168.xxx.xxx:9000/test/result.txt

[马阳阳]

我们遇到过因为jar包冲突导致task
manager无法启动，从而一直处于accepted状态的情况。这种情况下可以使用yarn logs
-applicationId <appId>查看一下日志，可以发现更多线索。

Fei Han wrote on 2020/2/9 17:34:

> @all:
>     在Flink1.10中，用yarn模式无法提交作业。
>    提示如下：
> lease check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:26,318 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:26,570 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:26,822 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:27,074 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:27,326 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:27,578 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:27,831 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:28,083 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
> 2020-02-09 17:22:28,336 INFO  org.apache.flink.yarn.YarnClusterDescriptor                   - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
>
>
> 另外，我看了yarn资源足够。但就是提交不了。
> 我的作业提交命令如下：
>   ./bin/flink run -m yarn-cluster  -yjm 1024  -ytm 2048  /opt/flink-1.10/examples/batch/WordCount.jar  --input hdfs://192.168.xxx.xxx:9000/test/LICENSE  --output hdfs://192.168.xxx.xxx:9000/test/result.txt

--
Sent from Postbox <https://www.postbox-inc.com>