Flink Sql client on yarn 问题

[air23]

你好。我在sql client 配置的yarn  yarn日志报错如下
2020-10-09 14:17:37,721 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - --------------------------------------------------------------------------------
2020-10-09 14:17:37,726 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  Starting YarnJobClusterEntrypoint (Version: 1.11.1, Scala: 2.11, Rev:7eb514a, Date:2020-07-15T07:02:09+02:00)
2020-10-09 14:17:37,726 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] -  OS current user: yarn
2020-10-09 14:17:38,101 FATAL org.apache.hadoop.conf.Configuration                         [] - error parsing conf core-site.xml
java.io.FileNotFoundException: /etc/hadoop/conf.cloudera.yarn/core-site.xml (Permission denied)
        at java.io.FileInputStream.open0(Native Method) ~[?:1.8.0_231]
        at java.io.FileInputStream.open(FileInputStream.java:195) ~[?:1.8.0_231]
        at java.io.FileInputStream.<init>(FileInputStream.java:138) ~[?:1.8.0_231]
        at java.io.FileInputStream.<init>(FileInputStream.java:93) ~[?:1.8.0_231]
        at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90) ~[?:1.8.0_231]
        at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188) ~[?:1.8.0_231]
        at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2629) ~[hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2703) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2653) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2559) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1078) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1132) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1540) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.SecurityUtil.setConfigurationInternal(SecurityUtil.java:85) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:74) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:320) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:308) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:895) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:861) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:728) [hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_231]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_231]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_231]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231]
        at org.apache.flink.runtime.util.EnvironmentInformation.getHadoopUser(EnvironmentInformation.java:201) [flink-dist_2.11-1.11.1.jar:1.11.1]
        at org.apache.flink.runtime.util.EnvironmentInformation.logEnvironmentInfo(EnvironmentInformation.java:399) [flink-dist_2.11-1.11.1.jar:1.11.1]
        at org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint.main(YarnJobClusterEntrypoint.java:67) [flink-dist_2.11-1.11.1.jar:1.11.1]
2020-10-09 14:17:38,113 WARN  org.apache.flink.runtime.util.EnvironmentInformation         [] - Error while accessing user/group information via Hadoop utils.
java.lang.reflect.InvocationTargetException: null
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_231]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_231]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_231]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231]
        at org.apache.flink.runtime.util.EnvironmentInformation.getHadoopUser(EnvironmentInformation.java:201) [flink-dist_2.11-1.11.1.jar:1.11.1]
        at org.apache.flink.runtime.util.EnvironmentInformation.logEnvironmentInfo(EnvironmentInformation.java:399) [flink-dist_2.11-1.11.1.jar:1.11.1]
        at org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint.main(YarnJobClusterEntrypoint.java:67) [flink-dist_2.11-1.11.1.jar:1.11.1]
Caused by: java.lang.ExceptionInInitializerError
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:320) ~[hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:308) ~[hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:895) ~[hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:861) ~[hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:728) ~[hadoop-common-2.6.0-cdh5.16.2.jar:?]
        ... 7 more
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /etc/hadoop/conf.cloudera.yarn/core-site.xml (Permission denied)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2811) ~[hadoop-common-2.6.0-cdh5.16.2.jar:?]
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2653) ~[hadoop-common-2.6.0-cdh5.16.2.jar:?]












sql 客户端显示如下




2020-10-09 14:17:35,194 INFO  org.apache.flink.yarn.YarnClusterDescriptor                  [] - Deploying cluster, current state ACCEPTED
[ERROR] Could not execute SQL statement. Reason:
org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment.
Diagnostics from YARN: Application application_1600321146023_0027 failed 1 times due to AM Container for appattempt_1600321146023_0027_000001 exited with  exitCode: 1
For more detailed output, check application tracking page:http://zt72:8088/proxy/application_1600321146023_0027/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1600321146023_0027_01_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:604)
        at org.apache.hadoop.util.Shell.run(Shell.java:507)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:789)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)


Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
If log aggregation is enabled on your cluster, use this command to further investigate the issue:
yarn logs -applicationId application_1600321146023_0027

[caozhen]

看错误提示是没权限读取core-site.xml，有没有检查core-site.xml的文件权限



--
Sent from: http://apache-flink.147419.n8.nabble.com/