Apache Flink 中文用户邮件列表

Scala REPL YARN 运行模式报 NoSuchMethodError

Classic

List

Threaded

3 messages Options

卢国庆

Scala REPL YARN 运行模式报 NoSuchMethodError

各位好，辛苦帮忙看个问题

使用 Scala REPL Yarn 运行模式报 NoSuchMethodError，截图如下：

$ ./bin/start-scala-shell.sh yarn

环境说明

CDH 5.16.2

测试 Flink 1.10.2 和 1.11.2 都能复现该问题

已分析内容

使用 Arthas 查看已加载类，加载的是 CDH 相关依赖

删除这个 CDH 依赖 jline-2.11.jar，不再报 NoSuchMethodError。但 Arthas 没有找到 jline.console.completer.CandidateListCompletionHandler，而是找到 scala.tools.jline_embedded.console.completer.CandidateListCompletionHandler，详见下面截图

Jacob

Re: Scala REPL YARN 运行模式报 NoSuchMethodError

hi，
你的截图好像没有上传成功，通过你的描述，大概是NoSuchMethod之类的错误，我前几天在升级flink版本时候也遇到过类似问题，后来的解决方案是
导入hadoop classpath （export HADOOP_CLASSPATH=`hadoop
classpath`）解决的，如果没有解决你的问题，尝试把flink-shaded-hadoop-2-uber*-*.jar放在 flink/lib下面

Thanks!
Jacob

--
Sent from: http://apache-flink.147419.n8.nabble.com/

Thanks!
Jacob

卢国庆

Re:Re: Scala REPL YARN 运行模式报 NoSuchMethodError

谢谢回复。

我已经采用 `hadoop classpath` 方式完成了 Hadoop 的集成。当前的问题是在 CDH 5.16.2 + Flink 环境下遇到的

补充下丢失的截图信息

使用 Scala REPL Yarn 运行模式报 NoSuchMethodError，详细错误信息如下：
$ ./bin/start-scala-shell.sh yarn
|
scala> Exception in thread "main" java.lang.NoSuchMethodError: jline.console.completer.CandidateListCompletionHandler.setPrintSpaceAfterFullCompletion(Z)V
at scala.tools.nsc.interpreter.jline.JLineConsoleReader.initCompletion(JLineReader.scala:139)
at scala.tools.nsc.interpreter.jline.InteractiveReader.postInit(JLineReader.scala:54)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1$$anonfun$25.apply(ILoop.scala:899)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1$$anonfun$25.apply(ILoop.scala:897)
at scala.tools.nsc.interpreter.SplashReader.postInit(InteractiveReader.scala:130)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1$$anonfun$scala$tools$nsc$interpreter$ILoop$$anonfun$$loopPostInit$1$1.apply$mcV$sp(ILoop.scala:926)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1$$anonfun$scala$tools$nsc$interpreter$ILoop$$anonfun$$loopPostInit$1$1.apply(ILoop.scala:908)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1$$anonfun$scala$tools$nsc$interpreter$ILoop$$anonfun$$loopPostInit$1$1.apply(ILoop.scala:908)
at scala.tools.nsc.interpreter.ILoop$$anonfun$mumly$1.apply(ILoop.scala:189)
at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:221)
at scala.tools.nsc.interpreter.ILoop.mumly(ILoop.scala:186)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1$$anonfun$startup$1$1.apply(ILoop.scala:979)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:990)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:891)
at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:891)
at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:891)
at org.apache.flink.api.scala.FlinkShell$.startShell(FlinkShell.scala:187)
at org.apache.flink.api.scala.FlinkShell$.main(FlinkShell.scala:131)
at org.apache.flink.api.scala.FlinkShell.main(FlinkShell.scala)
|

环境说明
CDH 5.16.2
测试 Flink 1.10.2 和 1.11.2 都能复现该问题

已分析内容
使用 Arthas 查看已加载类，加载的是 CDH 相关依赖
|
$ sc *CandidateListCompletionHandler
jline.console.completer.CandidateListCompletionHandler
Affect(row-cnt:1) cost in 113 ms.
[arthas@23856]$ sc -d jline.console.completer.CandidateListCompletionHandler
class-info jline.console.completer.CandidateListCompletionHandler
code-source /opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/jars/jline-2.11.jar
name jline.console.completer.CandidateListCompletionHandler
isInterface false
isAnnotation false
isEnum false
isAnonymousClass false
isArray false
isLocalClass false
isMemberClass false
isPrimitive false
isSynthetic false
simple-name CandidateListCompletionHandler
modifier public
annotation
interfaces jline.console.completer.CompletionHandler
super-class +-java.lang.Object
class-loader +-sun.misc.Launcher$AppClassLoader@677327b6
+-sun.misc.Launcher$ExtClassLoader@fcd6521
classLoaderHash 677327b6

Affect(row-cnt:1) cost in 39 ms.
|

删除这个 CDH 依赖 jline-2.11.jar，不再报 NoSuchMethodError。但 Arthas 没有找到 jline.console.completer.CandidateListCompletionHandler，而是找到 scala.tools.jline_embedded.console.completer.CandidateListCompletionHandler，详见下面截图

|
$ sc *CandidateListCompletionHandler
scala.tools.jline_embedded.console.completer.CandidateListCompletionHandler
Affect(row-cnt:1) cost in 47 ms.
[arthas@17882]$ sc -d scala.tools.jline_embedded.console.completer.CandidateListCompletionHandler
class-info scala.tools.jline_embedded.console.completer.CandidateListCompletionHandler
code-source /home/flink/luguoqing/flink/flink-1.10.2/lib/flink-dist_2.11-1.10.2.jar
name scala.tools.jline_embedded.console.completer.CandidateListCompletionHandler
isInterface false
isAnnotation false
isEnum false
isAnonymousClass false
isArray false
isLocalClass false
isMemberClass false
isPrimitive false
isSynthetic false
simple-name CandidateListCompletionHandler
modifier public
annotation
interfaces scala.tools.jline_embedded.console.completer.CompletionHandler
super-class +-java.lang.Object
class-loader +-sun.misc.Launcher$AppClassLoader@12a3a380
+-sun.misc.Launcher$ExtClassLoader@2eda0940
classLoaderHash 12a3a380

Affect(row-cnt:1) cost in 27 ms.
|

在 2020-12-16 18:12:57，"Jacob" <[hidden email]> 写道：

>hi，
>你的截图好像没有上传成功，通过你的描述，大概是NoSuchMethod之类的错误，我前几天在升级flink版本时候也遇到过类似问题，后来的解决方案是
>导入hadoop classpath （export HADOOP_CLASSPATH=`hadoop
>classpath`）解决的，如果没有解决你的问题，尝试把flink-shaded-hadoop-2-uber*-*.jar放在 flink/lib下面
>
>
>
>
>Thanks!
>Jacob
>
>
>
>--
>Sent from: http://apache-flink.147419.n8.nabble.com/