flink1.12 Standalone模式发送python脚本任务报错： java.lang.ClassNotFoundException: org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat

[肖越]

flink1.12.2 部署standalone集群模式，任务是pyflink实现链接Mysql数据库完成计算任务:

1. 已在 /user/local/flink-1.12.2/lib目录下，添加相关依赖：

    mysql-connector-java-8.0.12.jar，

    flink-connector-jdbc_2.11-1.12.0.jar，

    flink-table-api-java-1.12.0.jar

2.发送任务命令：

   bin/flink run  -py ../test.py -p 8

3.附报错信息如下；在线等路过部署过的大佬，指点一下~ 谢谢！

Traceback (most recent call last):

  File "../test.py", line 57, in <module>

    env.execute('Test')

  File "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/pyflink.zip/pyflink/table/table_environment.py", line 1276, in execute

  File "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", line 1286, in __call__

  File "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/pyflink.zip/pyflink/util/exceptions.py", line 147, in deco

  File "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/py4j-0.10.8.1-src.zip/py4j/protocol.py", line 328, in get_return_value

py4j.protocol.Py4JJavaError: An error occurred while calling o5.execute.

: org.apache.flink.util.FlinkException: Failed to execute job 'Pyflink1.12_Query_Time_Test'.

        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1918)

        at org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:135)

        at org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:76)

        at org.apache.flink.table.planner.delegation.ExecutorBase.execute(ExecutorBase.java:50)

        at org.apache.flink.table.api.internal.TableEnvironmentImpl.execute(TableEnvironmentImpl.java:1277)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)

        at org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)

        at org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)

        at org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)

        at org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)

        at org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)

        at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.RuntimeException: org.apache.flink.runtime.client.JobInitializationException: Could not instantiate JobManager.

        at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:316)

        at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75)

        at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)

        at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)

        at java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:457)

        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)

        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)

        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)

        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Caused by: org.apache.flink.runtime.client.JobInitializationException: Could not instantiate JobManager.

        at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:494)

        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

        at java.lang.Thread.run(Thread.java:748)

Caused by: org.apache.flink.runtime.client.JobExecutionException: Cannot initialize task 'Source: TableSourceScan(table=[[default_catalog, default_database, TP_GL_DAY, project=[DAY_ID]]], fields=[DAY_ID])': Loading the input/output formats failed:

        at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:239)

        at org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:322)

        at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:276)

        at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:249)

        at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:133)

        at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:111)

        at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345)

        at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:330)

        at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:95)

        at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:39)

        at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:162)

        at org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:86)

        at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:478)

        ... 4 more

Caused by: java.lang.Exception: Loading the input/output formats failed:

        at org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initInputOutputformatContainer(InputOutputFormatVertex.java:172)

        at org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initializeOnMaster(InputOutputFormatVertex.java:57)

        at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:235)

        ... 16 more

Caused by: java.lang.RuntimeException: Deserializing the input/output formats failed: Could not read the user code wrapper: org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat

        at org.apache.flink.runtime.jobgraph.InputOutputFormatContainer.<init>(InputOutputFormatContainer.java:69)

        at org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initInputOutputformatContainer(InputOutputFormatVertex.java:168)

        ... 18 more

Caused by: org.apache.flink.runtime.operators.util.CorruptConfigurationException: Could not read the user code wrapper: org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat

        at org.apache.flink.runtime.operators.util.TaskConfig.getStubWrapper(TaskConfig.java:307)

        at org.apache.flink.runtime.jobgraph.InputOutputFormatContainer.<init>(InputOutputFormatContainer.java:66)

        ... 19 more

Caused by: java.lang.ClassNotFoundException: org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat

        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)

        at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)

        at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:65)

        at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)

        at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172)

        at java.lang.Class.forName0(Native Method)

        at java.lang.Class.forName(Class.java:348)

        at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76)

        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1924)

        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1807)

        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2098)

        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)

        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)

        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)

        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)

        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)

        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)

        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)

        at java.util.HashMap.readObject(HashMap.java:1412)

        at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1185)

        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2234)

        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)

        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)

        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)

        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)

        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)

        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)

        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)

        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)

        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)

        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)

        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)

        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)

        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)

        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)

        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)

        at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541)

        at org.apache.flink.runtime.operators.util.TaskConfig.getStubWrapper(TaskConfig.java:304)

        ... 20 more




org.apache.flink.client.program.ProgramAbortException

        at org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:124)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:349)

        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:219)

        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)

        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)

        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)

        at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)

        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:422)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)

        at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)

        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)

[Dian Fu]

可以看一下：
https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/python/table-api-users-guide/dependency_management.html#java-dependency-in-python-program

flink-connector-jdbc_2.11-1.12.0.jar和mysql-connector-java-8.0.12.jar，需要放到PyFlink可以找到的地方。

On Mon, Mar 22, 2021 at 1:43 PM xiaoyue <[hidden email]> wrote:

> flink1.12.2 部署standalone集群模式，任务是pyflink实现链接Mysql数据库完成计算任务:
>
> 1. 已在 /user/local/flink-1.12.2/lib目录下，添加相关依赖：
>
>     mysql-connector-java-8.0.12.jar，
>
>     flink-connector-jdbc_2.11-1.12.0.jar，
>
>     flink-table-api-java-1.12.0.jar
>
> 2.发送任务命令：
>
>    bin/flink run  -py ../test.py -p 8
>
> 3.附报错信息如下；在线等路过部署过的大佬，指点一下~ 谢谢！
>
> Traceback (most recent call last):
>
>   File "../test.py", line 57, in <module>
>
>     env.execute('Test')
>
>   File
> "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/pyflink.zip/pyflink/table/table_environment.py",
> line 1276, in execute
>
>   File
> "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/py4j-0.10.8.1-src.zip/py4j/java_gateway.py",
> line 1286, in __call__
>
>   File
> "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/pyflink.zip/pyflink/util/exceptions.py",
> line 147, in deco
>
>   File
> "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/py4j-0.10.8.1-src.zip/py4j/protocol.py",
> line 328, in get_return_value
>
> py4j.protocol.Py4JJavaError: An error occurred while calling o5.execute.
>
> : org.apache.flink.util.FlinkException: Failed to execute job
> 'Pyflink1.12_Query_Time_Test'.
>
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1918)
>
>         at
> org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:135)
>
>         at
> org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:76)
>
>         at
> org.apache.flink.table.planner.delegation.ExecutorBase.execute(ExecutorBase.java:50)
>
>         at
> org.apache.flink.table.api.internal.TableEnvironmentImpl.execute(TableEnvironmentImpl.java:1277)
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>         at java.lang.reflect.Method.invoke(Method.java:498)
>
>         at
> org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
>
>         at
> org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
>
>         at
> org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
>
>         at
> org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
>
>         at
> org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
>
>         at
> org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
>
>         at java.lang.Thread.run(Thread.java:748)
>
> Caused by: java.lang.RuntimeException:
> org.apache.flink.runtime.client.JobInitializationException: Could not
> instantiate JobManager.
>
>         at
> org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:316)
>
>         at
> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75)
>
>         at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
>
>         at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
>
>         at
> java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:457)
>
>         at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
>
>         at
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
>
>         at
> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
>
>         at
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
>
> Caused by: org.apache.flink.runtime.client.JobInitializationException:
> Could not instantiate JobManager.
>
>         at
> org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:494)
>
>         at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
>
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
>         at java.lang.Thread.run(Thread.java:748)
>
> Caused by: org.apache.flink.runtime.client.JobExecutionException: Cannot
> initialize task 'Source: TableSourceScan(table=[[default_catalog,
> default_database, TP_GL_DAY, project=[DAY_ID]]], fields=[DAY_ID])': Loading
> the input/output formats failed:
>
>         at
> org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:239)
>
>         at
> org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:322)
>
>         at
> org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:276)
>
>         at
> org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:249)
>
>         at
> org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:133)
>
>         at
> org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:111)
>
>         at
> org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345)
>
>         at
> org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:330)
>
>         at
> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:95)
>
>         at
> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:39)
>
>         at
> org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:162)
>
>         at
> org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:86)
>
>         at
> org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:478)
>
>         ... 4 more
>
> Caused by: java.lang.Exception: Loading the input/output formats failed:
>
>         at
> org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initInputOutputformatContainer(InputOutputFormatVertex.java:172)
>
>         at
> org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initializeOnMaster(InputOutputFormatVertex.java:57)
>
>         at
> org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:235)
>
>         ... 16 more
>
> Caused by: java.lang.RuntimeException: Deserializing the input/output
> formats failed: Could not read the user code wrapper:
> org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat
>
>         at
> org.apache.flink.runtime.jobgraph.InputOutputFormatContainer.<init>(InputOutputFormatContainer.java:69)
>
>         at
> org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initInputOutputformatContainer(InputOutputFormatVertex.java:168)
>
>         ... 18 more
>
> Caused by:
> org.apache.flink.runtime.operators.util.CorruptConfigurationException:
> Could not read the user code wrapper:
> org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat
>
>         at
> org.apache.flink.runtime.operators.util.TaskConfig.getStubWrapper(TaskConfig.java:307)
>
>         at
> org.apache.flink.runtime.jobgraph.InputOutputFormatContainer.<init>(InputOutputFormatContainer.java:66)
>
>         ... 19 more
>
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat
>
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
>
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
>
>         at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
>
>         at
> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:65)
>
>         at
> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
>
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
>
>         at
> org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172)
>
>         at java.lang.Class.forName0(Native Method)
>
>         at java.lang.Class.forName(Class.java:348)
>
>         at
> org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76)
>
>         at
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1924)
>
>         at
> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1807)
>
>         at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2098)
>
>         at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>
>         at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)
>
>         at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)
>
>         at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)
>
>         at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
>
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
>
>         at java.util.HashMap.readObject(HashMap.java:1412)
>
>         at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>         at java.lang.reflect.Method.invoke(Method.java:498)
>
>         at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1185)
>
>         at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2234)
>
>         at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)
>
>         at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>
>         at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)
>
>         at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)
>
>         at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)
>
>         at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>
>         at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)
>
>         at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)
>
>         at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)
>
>         at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
>
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
>
>         at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)
>
>         at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)
>
>         at
> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)
>
>         at
> org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541)
>
>         at
> org.apache.flink.runtime.operators.util.TaskConfig.getStubWrapper(TaskConfig.java:304)
>
>         ... 20 more
>
>
>
>
> org.apache.flink.client.program.ProgramAbortException
>
>         at
> org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:124)
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>         at java.lang.reflect.Method.invoke(Method.java:498)
>
>         at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:349)
>
>         at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:219)
>
>         at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
>
>         at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
>
>         at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
>
>         at
> org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
>
>         at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
>
>         at java.security.AccessController.doPrivileged(Native Method)
>
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
>
>         at
> org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>
>         at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)

[肖越]

好的，问题已经解决~ 谢谢您！










在 2021-03-22 16:50:03，"Dian Fu" <[hidden email]> 写道：

>可以看一下：
>https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/python/table-api-users-guide/dependency_management.html#java-dependency-in-python-program
>
>flink-connector-jdbc_2.11-1.12.0.jar和mysql-connector-java-8.0.12.jar，需要放到PyFlink可以找到的地方。
>
>On Mon, Mar 22, 2021 at 1:43 PM xiaoyue <[hidden email]> wrote:
>
>> flink1.12.2 部署standalone集群模式，任务是pyflink实现链接Mysql数据库完成计算任务:
>>
>> 1. 已在 /user/local/flink-1.12.2/lib目录下，添加相关依赖：
>>
>>     mysql-connector-java-8.0.12.jar，
>>
>>     flink-connector-jdbc_2.11-1.12.0.jar，
>>
>>     flink-table-api-java-1.12.0.jar
>>
>> 2.发送任务命令：
>>
>>    bin/flink run  -py ../test.py -p 8
>>
>> 3.附报错信息如下；在线等路过部署过的大佬，指点一下~ 谢谢！
>>
>> Traceback (most recent call last):
>>
>>   File "../test.py", line 57, in <module>
>>
>>     env.execute('Test')
>>
>>   File
>> "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/pyflink.zip/pyflink/table/table_environment.py",
>> line 1276, in execute
>>
>>   File
>> "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/py4j-0.10.8.1-src.zip/py4j/java_gateway.py",
>> line 1286, in __call__
>>
>>   File
>> "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/pyflink.zip/pyflink/util/exceptions.py",
>> line 147, in deco
>>
>>   File
>> "/usr/local/env/flink1.12_py3_env/flink-1.12.2/opt/python/py4j-0.10.8.1-src.zip/py4j/protocol.py",
>> line 328, in get_return_value
>>
>> py4j.protocol.Py4JJavaError: An error occurred while calling o5.execute.
>>
>> : org.apache.flink.util.FlinkException: Failed to execute job
>> 'Pyflink1.12_Query_Time_Test'.
>>
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1918)
>>
>>         at
>> org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:135)
>>
>>         at
>> org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:76)
>>
>>         at
>> org.apache.flink.table.planner.delegation.ExecutorBase.execute(ExecutorBase.java:50)
>>
>>         at
>> org.apache.flink.table.api.internal.TableEnvironmentImpl.execute(TableEnvironmentImpl.java:1277)
>>
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>>         at java.lang.reflect.Method.invoke(Method.java:498)
>>
>>         at
>> org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
>>
>>         at
>> org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
>>
>>         at
>> org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
>>
>>         at
>> org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
>>
>>         at
>> org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
>>
>>         at
>> org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
>>
>>         at java.lang.Thread.run(Thread.java:748)
>>
>> Caused by: java.lang.RuntimeException:
>> org.apache.flink.runtime.client.JobInitializationException: Could not
>> instantiate JobManager.
>>
>>         at
>> org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:316)
>>
>>         at
>> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75)
>>
>>         at
>> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
>>
>>         at
>> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
>>
>>         at
>> java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:457)
>>
>>         at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
>>
>>         at
>> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
>>
>>         at
>> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
>>
>>         at
>> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
>>
>> Caused by: org.apache.flink.runtime.client.JobInitializationException:
>> Could not instantiate JobManager.
>>
>>         at
>> org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:494)
>>
>>         at
>> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
>>
>>         at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>
>>         at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>
>>         at java.lang.Thread.run(Thread.java:748)
>>
>> Caused by: org.apache.flink.runtime.client.JobExecutionException: Cannot
>> initialize task 'Source: TableSourceScan(table=[[default_catalog,
>> default_database, TP_GL_DAY, project=[DAY_ID]]], fields=[DAY_ID])': Loading
>> the input/output formats failed:
>>
>>         at
>> org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:239)
>>
>>         at
>> org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:322)
>>
>>         at
>> org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:276)
>>
>>         at
>> org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:249)
>>
>>         at
>> org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:133)
>>
>>         at
>> org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:111)
>>
>>         at
>> org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345)
>>
>>         at
>> org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:330)
>>
>>         at
>> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:95)
>>
>>         at
>> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:39)
>>
>>         at
>> org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:162)
>>
>>         at
>> org.apache.flink.runtime.dispatcher.DefaultJobManagerRunnerFactory.createJobManagerRunner(DefaultJobManagerRunnerFactory.java:86)
>>
>>         at
>> org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$5(Dispatcher.java:478)
>>
>>         ... 4 more
>>
>> Caused by: java.lang.Exception: Loading the input/output formats failed:
>>
>>         at
>> org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initInputOutputformatContainer(InputOutputFormatVertex.java:172)
>>
>>         at
>> org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initializeOnMaster(InputOutputFormatVertex.java:57)
>>
>>         at
>> org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:235)
>>
>>         ... 16 more
>>
>> Caused by: java.lang.RuntimeException: Deserializing the input/output
>> formats failed: Could not read the user code wrapper:
>> org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat
>>
>>         at
>> org.apache.flink.runtime.jobgraph.InputOutputFormatContainer.<init>(InputOutputFormatContainer.java:69)
>>
>>         at
>> org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initInputOutputformatContainer(InputOutputFormatVertex.java:168)
>>
>>         ... 18 more
>>
>> Caused by:
>> org.apache.flink.runtime.operators.util.CorruptConfigurationException:
>> Could not read the user code wrapper:
>> org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat
>>
>>         at
>> org.apache.flink.runtime.operators.util.TaskConfig.getStubWrapper(TaskConfig.java:307)
>>
>>         at
>> org.apache.flink.runtime.jobgraph.InputOutputFormatContainer.<init>(InputOutputFormatContainer.java:66)
>>
>>         ... 19 more
>>
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flink.connector.jdbc.table.JdbcRowDataInputFormat
>>
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
>>
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
>>
>>         at
>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
>>
>>         at
>> org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:65)
>>
>>         at
>> org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
>>
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
>>
>>         at
>> org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172)
>>
>>         at java.lang.Class.forName0(Native Method)
>>
>>         at java.lang.Class.forName(Class.java:348)
>>
>>         at
>> org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76)
>>
>>         at
>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1924)
>>
>>         at
>> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1807)
>>
>>         at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2098)
>>
>>         at
>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>>
>>         at
>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)
>>
>>         at
>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)
>>
>>         at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)
>>
>>         at
>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>>
>>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
>>
>>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
>>
>>         at java.util.HashMap.readObject(HashMap.java:1412)
>>
>>         at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
>>
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>>         at java.lang.reflect.Method.invoke(Method.java:498)
>>
>>         at
>> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1185)
>>
>>         at
>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2234)
>>
>>         at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)
>>
>>         at
>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>>
>>         at
>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)
>>
>>         at
>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)
>>
>>         at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)
>>
>>         at
>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>>
>>         at
>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2343)
>>
>>         at
>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2267)
>>
>>         at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2125)
>>
>>         at
>> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1624)
>>
>>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
>>
>>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
>>
>>         at
>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)
>>
>>         at
>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)
>>
>>         at
>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)
>>
>>         at
>> org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541)
>>
>>         at
>> org.apache.flink.runtime.operators.util.TaskConfig.getStubWrapper(TaskConfig.java:304)
>>
>>         ... 20 more
>>
>>
>>
>>
>> org.apache.flink.client.program.ProgramAbortException
>>
>>         at
>> org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:124)
>>
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>>         at java.lang.reflect.Method.invoke(Method.java:498)
>>
>>         at
>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:349)
>>
>>         at
>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:219)
>>
>>         at
>> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
>>
>>         at
>> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
>>
>>         at
>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
>>
>>         at
>> org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
>>
>>         at
>> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
>>
>>         at java.security.AccessController.doPrivileged(Native Method)
>>
>>         at javax.security.auth.Subject.doAs(Subject.java:422)
>>
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
>>
>>         at
>> org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>>
>>         at
>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)