Apache Flink 中文用户邮件列表

flink读取mongo数据本地部署成功，flink-cluster部署找不到类

classic Classic list List threaded Threaded
♦
♦
 3 messages Options
GeGe
Reply | Threaded
Open this post in threaded view
♦
♦
|

flink读取mongo数据本地部署成功，flink-cluster部署找不到类

GeGe


您好！


我用flink从mongo读取数据，在本地运行成功，但是部署到本地flink-cluster中却报错：



java.lang.NoClassDefFoundError: com/mongodb/MongoClient

at java.lang.Class.getDeclaredMethods0(Native Method)

at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)

at java.lang.Class.getDeclaredMethod(Class.java:2128)

at org.apache.flink.api.java.ClosureCleaner.usesCustomSerialization(ClosureCleaner.java:164)

at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:89)

at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:71)

at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1901)

at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1614)

at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1571)

at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1553)

at com.test.flink.service.MongoFlinkMainService.main(MongoFlinkMainService.java:23)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)

at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)

at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149)

at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:699)

at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:232)

at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916)

at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992)

at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)

at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)

Caused by: java.lang.ClassNotFoundException: com.mongodb.MongoClient

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:61)

at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 24 more





之前得到的数据是org.bson.Document格式，但是报错：找不到Document，所以就换成了String类型。现在报错的类也是在同一个jar包下：mongo-driver-java-3.8.2.jar ,换了各种版本，解压maven的jar包查看也有这个jar包。现在就是不知道为什么找不到这个jar包下的任何类。


请求各位大神帮帮忙~~非常非常感谢~~


Best wishes,


Gege
| |
Ge Ge
Software Engineer
|
|
[hidden email]
|
签名由网易邮箱大师定制

zilong xiao
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: flink读取mongo数据本地部署成功，flink-cluster部署找不到类

zilong xiao
有可能是依赖冲突了，可以尝试用maven shade jar看看

GeGe <[hidden email]> 于2020年9月25日周五 下午3:54写道：

>
>
> 您好！
>
>
> 我用flink从mongo读取数据，在本地运行成功，但是部署到本地flink-cluster中却报错：
>
>
>
> java.lang.NoClassDefFoundError: com/mongodb/MongoClient
>
> at java.lang.Class.getDeclaredMethods0(Native Method)
>
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
>
> at java.lang.Class.getDeclaredMethod(Class.java:2128)
>
> at
> org.apache.flink.api.java.ClosureCleaner.usesCustomSerialization(ClosureCleaner.java:164)
>
> at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:89)
>
> at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:71)
>
> at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1901)
>
> at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1614)
>
> at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1571)
>
> at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1553)
>
> at
> com.test.flink.service.MongoFlinkMainService.main(MongoFlinkMainService.java:23)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)
>
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)
>
> at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149)
>
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:699)
>
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:232)
>
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916)
>
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992)
>
> at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)
>
> Caused by: java.lang.ClassNotFoundException: com.mongodb.MongoClient
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>
> at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:61)
>
> at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>
> ... 24 more
>
>
>
>
>
> 之前得到的数据是org.bson.Document格式，但是报错：找不到Document，所以就换成了String类型。现在报错的类也是在同一个jar包下：mongo-driver-java-3.8.2.jar
> ,换了各种版本，解压maven的jar包查看也有这个jar包。现在就是不知道为什么找不到这个jar包下的任何类。
>
>
> 请求各位大神帮帮忙~~非常非常感谢~~
>
>
> Best wishes,
>
>
> Gege
> | |
> Ge Ge
> Software Engineer
> |
> |
> [hidden email]
> |
> 签名由网易邮箱大师定制
>
>
GeGe
Reply | Threaded
Open this post in threaded view
♦
♦
|

回复： flink读取mongo数据本地部署成功，flink-cluster部署找不到类

GeGe
您好！


十分感谢您的回答！之前有进行尝试，但是还是没有解决这个问题。最后觉得应该是jar包不能读取到，所以把那个jar包单独放到了flink的lib下面，运行成功了。还没有找到一劳永逸解决的办法，不过十分感谢您的回复，谢谢~


Best wishes,


Gege


| |
Ge Ge
Software Engineer
|
|
[hidden email]
|
签名由网易邮箱大师定制


在2020年09月25日 16:01，zilong xiao<[hidden email]> 写道：
有可能是依赖冲突了，可以尝试用maven shade jar看看

GeGe <[hidden email]> 于2020年9月25日周五 下午3:54写道：



您好！


我用flink从mongo读取数据，在本地运行成功，但是部署到本地flink-cluster中却报错：



java.lang.NoClassDefFoundError: com/mongodb/MongoClient

at java.lang.Class.getDeclaredMethods0(Native Method)

at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)

at java.lang.Class.getDeclaredMethod(Class.java:2128)

at
org.apache.flink.api.java.ClosureCleaner.usesCustomSerialization(ClosureCleaner.java:164)

at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:89)

at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:71)

at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.clean(StreamExecutionEnvironment.java:1901)

at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1614)

at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1571)

at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.java:1553)

at
com.test.flink.service.MongoFlinkMainService.main(MongoFlinkMainService.java:23)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)

at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)

at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149)

at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:699)

at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:232)

at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916)

at
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992)

at
org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)

at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)

Caused by: java.lang.ClassNotFoundException: com.mongodb.MongoClient

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at
org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:61)

at
org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 24 more





之前得到的数据是org.bson.Document格式，但是报错：找不到Document，所以就换成了String类型。现在报错的类也是在同一个jar包下：mongo-driver-java-3.8.2.jar
,换了各种版本，解压maven的jar包查看也有这个jar包。现在就是不知道为什么找不到这个jar包下的任何类。


请求各位大神帮帮忙~~非常非常感谢~~


Best wishes,


Gege
| |
Ge Ge
Software Engineer
|
|
[hidden email]
|
签名由网易邮箱大师定制


Free forum by Nabble Edit this page