Apache Flink 中文用户邮件列表

Flink yarn session exception

classic Classic list List threaded Threaded
♦
♦
 5 messages Options
Rainie Li
Reply | Threaded
Open this post in threaded view
♦
♦
|

Flink yarn session exception

Rainie Li
大佬们好，我是flink新手，正在用flink 1.9.1
Flink APP cannot run, APP log error,  想求教一下会是什么原因造成的，多谢🙏

2020-06-16 17:06:21,921 WARN  org.apache.flink.client.cli.CliFrontend
                - Could not load CLI class
org.apache.flink.yarn.cli.FlinkYarnSessionCli.
java.lang.NoClassDefFoundError:
org/apache/hadoop/yarn/exceptions/YarnException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1185)
at
org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1145)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1070)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.exceptions.YarnException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 5 more
2020-06-16 17:06:21,980 INFO  org.apache.flink.core.fs.FileSystem
                - Hadoop is not in the classpath/dependencies. The extended
set of supported File Systems via Hadoop is not available.
Paul Lam
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink yarn session exception

Paul Lam
日志里说得比较清楚了，classpath 里没有 Hadoop 的 lib。可以参考这个文档 [1] 来配置你的环境。

1. https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html <https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html>
Best,
Paul Lam

> 2020年7月16日 15:46，Rainie Li <[hidden email]> 写道：
>
> 大佬们好，我是flink新手，正在用flink 1.9.1
> Flink APP cannot run, APP log error,  想求教一下会是什么原因造成的，多谢🙏
>
> 2020-06-16 17:06:21,921 WARN  org.apache.flink.client.cli.CliFrontend
>                - Could not load CLI class
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.
> java.lang.NoClassDefFoundError:
> org/apache/hadoop/yarn/exceptions/YarnException
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1185)
> at
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1145)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1070)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.exceptions.YarnException
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 5 more
> 2020-06-16 17:06:21,980 INFO  org.apache.flink.core.fs.FileSystem
>                - Hadoop is not in the classpath/dependencies. The extended
> set of supported File Systems via Hadoop is not available.

Rainie Li
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink yarn session exception

Rainie Li
多谢，我set了这些envs:

export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export PATH=$JAVA_HOME/bin:$PATH
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_CLASSPATH=`hadoop classpath`
export FLINK_CONF_DIR=/etc/flink-1.9.1/conf
export FLINK_LOG_DIR=/home/rainieli/

有什么问题吗？🙏


On Thu, Jul 16, 2020 at 1:12 AM Paul Lam <[hidden email]> wrote:

> 日志里说得比较清楚了，classpath 里没有 Hadoop 的 lib。可以参考这个文档 [1] 来配置你的环境。
>
> 1.
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
> <
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
> >
> Best,
> Paul Lam
>
> > 2020年7月16日 15:46，Rainie Li <[hidden email]> 写道：
> >
> > 大佬们好，我是flink新手，正在用flink 1.9.1
> > Flink APP cannot run, APP log error,  想求教一下会是什么原因造成的，多谢🙏
> >
> > 2020-06-16 17:06:21,921 WARN  org.apache.flink.client.cli.CliFrontend
> >                - Could not load CLI class
> > org.apache.flink.yarn.cli.FlinkYarnSessionCli.
> > java.lang.NoClassDefFoundError:
> > org/apache/hadoop/yarn/exceptions/YarnException
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:264)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1185)
> > at
> >
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1145)
> > at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1070)
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.hadoop.yarn.exceptions.YarnException
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > ... 5 more
> > 2020-06-16 17:06:21,980 INFO  org.apache.flink.core.fs.FileSystem
> >                - Hadoop is not in the classpath/dependencies. The
> extended
> > set of supported File Systems via Hadoop is not available.
>
>
忝忝向仧
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink yarn session exception

忝忝向仧
In reply to this post by Rainie Li
你可以看看lib里面的包跟官网的要求是不是一样的



发自我的iPhone


------------------ Original ------------------
From: Rainie Li <[hidden email]&gt;
Date: Fri,Jul 17,2020 1:06 AM
To: user-zh <[hidden email]&gt;
Subject: Re: Flink yarn session exception



多谢，我set了这些envs:

export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export PATH=$JAVA_HOME/bin:$PATH
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_CLASSPATH=`hadoop classpath`
export FLINK_CONF_DIR=/etc/flink-1.9.1/conf
export FLINK_LOG_DIR=/home/rainieli/

有什么问题吗？🙏


On Thu, Jul 16, 2020 at 1:12 AM Paul Lam <[hidden email]&gt; wrote:

&gt; 日志里说得比较清楚了，classpath 里没有 Hadoop 的 lib。可以参考这个文档 [1] 来配置你的环境。
&gt;
&gt; 1.
&gt; https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
&gt; <
&gt; https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
&gt; &gt;
&gt; Best,
&gt; Paul Lam
&gt;
&gt; &gt; 2020年7月16日 15:46，Rainie Li <[hidden email]&gt; 写道：
&gt; &gt;
&gt; &gt; 大佬们好，我是flink新手，正在用flink 1.9.1
&gt; &gt; Flink APP cannot run, APP log error,&nbsp; 想求教一下会是什么原因造成的，多谢🙏
&gt; &gt;
&gt; &gt; 2020-06-16 17:06:21,921 WARN&nbsp; org.apache.flink.client.cli.CliFrontend
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Could not load CLI class
&gt; &gt; org.apache.flink.yarn.cli.FlinkYarnSessionCli.
&gt; &gt; java.lang.NoClassDefFoundError:
&gt; &gt; org/apache/hadoop/yarn/exceptions/YarnException
&gt; &gt; at java.lang.Class.forName0(Native Method)
&gt; &gt; at java.lang.Class.forName(Class.java:264)
&gt; &gt; at
&gt; &gt;
&gt; org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1185)
&gt; &gt; at
&gt; &gt;
&gt; org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1145)
&gt; &gt; at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1070)
&gt; &gt; Caused by: java.lang.ClassNotFoundException:
&gt; &gt; org.apache.hadoop.yarn.exceptions.YarnException
&gt; &gt; at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
&gt; &gt; at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
&gt; &gt; at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
&gt; &gt; at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
&gt; &gt; ... 5 more
&gt; &gt; 2020-06-16 17:06:21,980 INFO&nbsp; org.apache.flink.core.fs.FileSystem
&gt; &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; - Hadoop is not in the classpath/dependencies. The
&gt; extended
&gt; &gt; set of supported File Systems via Hadoop is not available.
&gt;
&gt;
Rainie Li
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Flink yarn session exception

Rainie Li
好搭，谢谢！

On Thu, Jul 16, 2020 at 5:32 PM 忝忝向仧 <[hidden email]> wrote:

> 你可以看看lib里面的包跟官网的要求是不是一样的
>
>
>
> 发自我的iPhone
>
>
> ------------------ Original ------------------
> From: Rainie Li <[hidden email]&gt;
> Date: Fri,Jul 17,2020 1:06 AM
> To: user-zh <[hidden email]&gt;
> Subject: Re: Flink yarn session exception
>
>
>
> 多谢，我set了这些envs:
>
> export JAVA_HOME=/usr/lib/jvm/java-8-oracle
> export PATH=$JAVA_HOME/bin:$PATH
> export HADOOP_HOME=/usr/local/hadoop
> export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
> export HADOOP_CLASSPATH=`hadoop classpath`
> export FLINK_CONF_DIR=/etc/flink-1.9.1/conf
> export FLINK_LOG_DIR=/home/rainieli/
>
> 有什么问题吗？🙏
>
>
> On Thu, Jul 16, 2020 at 1:12 AM Paul Lam <[hidden email]&gt; wrote:
>
> &gt; 日志里说得比较清楚了，classpath 里没有 Hadoop 的 lib。可以参考这个文档 [1] 来配置你的环境。
> &gt;
> &gt; 1.
> &gt;
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
> &gt; <
> &gt;
> https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html
> &gt; &gt;
> &gt; Best,
> &gt; Paul Lam
> &gt;
> &gt; &gt; 2020年7月16日 15:46，Rainie Li <[hidden email]&gt;
> 写道：
> &gt; &gt;
> &gt; &gt; 大佬们好，我是flink新手，正在用flink 1.9.1
> &gt; &gt; Flink APP cannot run, APP log error,&nbsp; 想求教一下会是什么原因造成的，多谢🙏
> &gt; &gt;
> &gt; &gt; 2020-06-16 17:06:21,921 WARN&nbsp;
> org.apache.flink.client.cli.CliFrontend
> &gt;
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> - Could not load CLI class
> &gt; &gt; org.apache.flink.yarn.cli.FlinkYarnSessionCli.
> &gt; &gt; java.lang.NoClassDefFoundError:
> &gt; &gt; org/apache/hadoop/yarn/exceptions/YarnException
> &gt; &gt; at java.lang.Class.forName0(Native Method)
> &gt; &gt; at java.lang.Class.forName(Class.java:264)
> &gt; &gt; at
> &gt; &gt;
> &gt;
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1185)
> &gt; &gt; at
> &gt; &gt;
> &gt;
> org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1145)
> &gt; &gt; at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1070)
> &gt; &gt; Caused by: java.lang.ClassNotFoundException:
> &gt; &gt; org.apache.hadoop.yarn.exceptions.YarnException
> &gt; &gt; at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> &gt; &gt; at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> &gt; &gt; at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> &gt; &gt; at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> &gt; &gt; ... 5 more
> &gt; &gt; 2020-06-16 17:06:21,980 INFO&nbsp;
> org.apache.flink.core.fs.FileSystem
> &gt;
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> - Hadoop is not in the classpath/dependencies. The
> &gt; extended
> &gt; &gt; set of supported File Systems via Hadoop is not available.
> &gt;
> &gt;
Free forum by Nabble Edit this page