Flink-1.11.1 Application-Mode提交测试

hi, everyone

当我把jar包都上传至hdfs时，使用如下命令进行application mode提交，

./bin/flink run-application -t yarn-application -Dyarn.provided.lib.dirs="hdfs:///user/flink/lib" -c com.yui.flink.demo.Kafka2Mysql hdfs:///user/flink/app_jars/kafka2mysql.jar

报异常如下：
------------------------------------------------------------
 The program finished with the following exception:

org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn Application Cluster
        at org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:414)
        at org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:64)
        at org.apache.flink.client.cli.CliFrontend.runApplication(CliFrontend.java:197)
        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:919)
        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
        at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)
Caused by: org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment.
Diagnostics from YARN: Application application_1598223665550_0009 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1598223665550_0009_000001 exited with  exitCode: -1
Failing this attempt.Diagnostics: [2020-08-25 15:12:48.975]Destination must be relative
For more detailed output, check the application tracking page: http://ck233:8088/cluster/app/application_1598223665550_0009 Then click on links to logs of each attempt.
. Failing the application.
If log aggregation is enabled on your cluster, use this command to further investigate the issue:
yarn logs -applicationId application_1598223665550_0009
        at org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:1021)
        at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:524)
        at org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:407)
        ... 9 more

其他没有任何的错误了，使用run -m yarn-cluster是可以正常提交的

best,
amenhub

[Yang Wang]

你给的这个报错目前并不能发现任何有效的信息，有可能的原因是在运行main的时候出错了
Application模式和perjob模式的很大的一个区别在于用户main运行的位置不一样

你把JobManager的log发一下，这样能看到更多详细的报错信息


Best,
Yang

[hidden email] <[hidden email]> 于2020年8月25日周二 下午3:29写道：

> hi, everyone
>
> 当我把jar包都上传至hdfs时，使用如下命令进行application mode提交，
>
> ./bin/flink run-application -t yarn-application
> -Dyarn.provided.lib.dirs="hdfs:///user/flink/lib" -c
> com.yui.flink.demo.Kafka2Mysql hdfs:///user/flink/app_jars/kafka2mysql.jar
>
> 报异常如下：
> ------------------------------------------------------------
>  The program finished with the following exception:
>
> org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't
> deploy Yarn Application Cluster
>         at
> org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:414)
>         at
> org.apache.flink.client.deployment.application.cli.ApplicationClusterDeployer.run(ApplicationClusterDeployer.java:64)
>         at
> org.apache.flink.client.cli.CliFrontend.runApplication(CliFrontend.java:197)
>         at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:919)
>         at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
>         at
> org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>         at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)
> Caused by:
> org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The
> YARN application unexpectedly switched to state FAILED during deployment.
> Diagnostics from YARN: Application application_1598223665550_0009 failed 1
> times (global limit =2; local limit is =1) due to AM Container for
> appattempt_1598223665550_0009_000001 exited with  exitCode: -1
> Failing this attempt.Diagnostics: [2020-08-25 15:12:48.975]Destination
> must be relative
> For more detailed output, check the application tracking page:
> http://ck233:8088/cluster/app/application_1598223665550_0009 Then click
> on links to logs of each attempt.
> . Failing the application.
> If log aggregation is enabled on your cluster, use this command to further
> investigate the issue:
> yarn logs -applicationId application_1598223665550_0009
>         at
> org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:1021)
>         at
> org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:524)
>         at
> org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:407)
>         ... 9 more
>
> 其他没有任何的错误了，使用run -m yarn-cluster是可以正常提交的
>
> best,
> amenhub
>