flink 任务动态编译异常

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

flink 任务动态编译异常

iwodetianna
hi
遇到了一个问题, 本地调测正常,扔服务器上就会报错

环境:
flink: 1.10.1
scala: 2.11

业务需求flink动态生成规则, 我用一个模板文件根据配置生成完整的规则, 然后动态编译,在Pattern的filter函数中调用

任务jar包放到了flink的lib目录下
请大神帮忙分析一下原因,谢谢!!!

报错信息:
Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: e1MyTest
ClassLoader info: URL ClassLoader:
    file: '/tmp/blobStore-d31dca76-0497-40a3-8e86-9df357cf90eb/job_2aa246bfd0d9e56ff448a8ecd1f57316/blob_p-07a73cda0b33b2be6e2db11c35dd37629fbe23f7-1f123a4f3e2d59eeddc0062b149e5abd' (valid JAR)
Class not resolvable through given classloader.
 at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:266)
 at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:115)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:432)
 at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:460)
 at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:708)
 at org.apache.flink.runtime.taskmanager.Task.run(Task.java:533)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: e1MyTest
 at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:348)
 at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
 at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
 at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
 at java.util.HashMap.readObject(HashMap.java:1412)
 at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1170)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2178)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
 at java.util.ArrayList.readObject(ArrayList.java:797)
 at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1170)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2178)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
 at java.util.ArrayList.readObject(ArrayList.java:797)
 at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1170)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2178)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
 at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:576)
 at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:562)
 at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:550)
 at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:511)
 at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:254)
 ... 6 more