flink12同时使用flink-connector-kafka和flink-sql-connector-kafka会引发类冲突问题

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

flink12同时使用flink-connector-kafka和flink-sql-connector-kafka会引发类冲突问题

site
在yarn中部署streaming程序,maven使用依赖
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka-2.2_2.11</artifactId>
<version>1.12.0</version>
</dependency>
在flink的lib目录中有flink-sql-connector-kafka_2.11-1.12.0.jar,因为类冲突问题会引起在yarn中程序部署失败,flink默认类加载机制child-first,改为parent-first也样,类冲突问题可以参考http://apache-flink.147419.n8.nabble.com/Kafka-Consumer-td3475.html
org.apache.kafka.common.KafkaException: org.apache.flink.kafka.shaded.org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer


不会提ISSUE,还请社区修复
Reply | Threaded
Open this post in threaded view
|

Re:flink12同时使用flink-connector-kafka和flink-sql-connector-kafka会引发类冲突问题

felixzh
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka-2.2_2.11</artifactId>
<version>1.12.0</version><scope>provided</scope></dependency>















在 2020-12-26 10:37:52,"site" <[hidden email]> 写道:

>在yarn中部署streaming程序,maven使用依赖
><dependency>
><groupId>org.apache.flink</groupId>
><artifactId>flink-connector-kafka-2.2_2.11</artifactId>
><version>1.12.0</version>
></dependency>
>在flink的lib目录中有flink-sql-connector-kafka_2.11-1.12.0.jar,因为类冲突问题会引起在yarn中程序部署失败,flink默认类加载机制child-first,改为parent-first也样,类冲突问题可以参考http://apache-flink.147419.n8.nabble.com/Kafka-Consumer-td3475.html
>org.apache.kafka.common.KafkaException: org.apache.flink.kafka.shaded.org.apache.kafka.common.serialization.ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer
>
>
>不会提ISSUE,还请社区修复