Flink SQL format问题

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Flink SQL format问题

guaishushu1103@163.com
csv.field-delimiter
csv.line-delimiter
想问下大佬们 官方文档说是可以设置这两个值,指定行分隔和字段分隔,但是在设置kafka sinkTable的时候会出现语法错误???很奇怪



[hidden email]
Reply | Threaded
Open this post in threaded view
|

回复:Flink SQL format问题

Shuai Xia
Hi,麻烦代码以及报错贴一下


------------------------------------------------------------------
发件人:[hidden email] <[hidden email]>
发送时间:2020年9月16日(星期三) 10:29
收件人:user-zh <[hidden email]>
主 题:Flink SQL format问题

csv.field-delimiter
csv.line-delimiter
想问下大佬们 官方文档说是可以设置这两个值,指定行分隔和字段分隔,但是在设置kafka sinkTable的时候会出现语法错误???很奇怪



[hidden email]
Reply | Threaded
Open this post in threaded view
|

回复:回复:Flink SQL format问题

Shuai Xia
U&'\\000A',代码也用了U&?


------------------------------------------------------------------
发件人:[hidden email] <[hidden email]>
发送时间:2020年9月16日(星期三) 10:50
收件人:Shuai Xia <[hidden email]>
主 题:Re: 回复:Flink SQL format问题



表定义:
'connector.type' = 'kafka',
'connector.version' = 'universal',
'connector.topic' = 'xxxx',
'connector.properties.bootstrap.servers' = 'xxxx',
'connector.properties.zookeeper.connect' = 'xxxx',
'connector.properties.group.id' = 'xxxx',
'connector.properties.client.id' = 'xxxx',
'connector.startup-mode' = 'latest-offset',
'format.type' = 'csv',
'csv.field-delimiter' = '\001',
'csv.line-delimiter' = U&'\\000A'
错误:
aused by: org.apache.flink.table.api.NoMatchingTableFactoryException: Could not find a suitable table factory for 'org.apache.flink.table.factories.TableSinkFactory' in (com.dataplatform.flink.util.FlinkDebugThread)
[2020-09-16 10:46:52,533] INFO ---the classpath. (com.dataplatform.flink.util.FlinkDebugThread)
bugThread)
[2020-09-16 10:46:52,533] INFO ---      at org.apache.flink.table.planner.delegation.PlannerBase$$anonfun$1.apply(PlannerBase.scala:163) (com.dataplatform.flink.util.Flink
DebugThread)
[2020-09-16 10:46:52,533] INFO ---      at org.apache.flink.table.planner.delegation.PlannerBase$$anonfun$1.apply(PlannerBase.scala:163) (com.dataplatform.flink.util.Flink
DebugThread)
[2020-09-16 10:46:52,533] INFO ---      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) (com.dataplatform.flink.util.FlinkDebugThread)
[2020-09-16 10:46:52,533] INFO ---      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) (com.dataplatform.flink.util.FlinkDebugThread)
[2020-09-16 10:46:52,533] INFO ---      at scala.collection.Iterator$class.foreach(Iterator.scala:891) (com.dataplatform.flink.util.FlinkDebugThread)
[2020-09-16 10:46:52,533] INFO ---      at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) (com.dataplatform.flink.util.FlinkDebugThread)
[2020-09-16 10:46:52,533] INFO ---      at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) (com.dataplatform.flink.util.FlinkDebugThread)
[2020-09-16 10:46:52,533] INFO ---      at scala.collection.AbstractIterable.foreach(Iterable.scala:54) (com.dataplatform.flink.util.FlinkDebugThread)




[hidden email]
发件人: Shuai Xia
发送时间: 2020-09-16 10:33
收件人: user-zh; guaishushu1103
主题: 回复:Flink SQL format问题
Hi,麻烦代码以及报错贴一下

------------------------------------------------------------------
发件人:[hidden email] <[hidden email]>
发送时间:2020年9月16日(星期三) 10:29
收件人:user-zh <[hidden email]>
主 题:Flink SQL format问题

csv.field-delimiter
csv.line-delimiter
想问下大佬们 官方文档说是可以设置这两个值,指定行分隔和字段分隔,但是在设置kafka sinkTable的时候会出现语法错误???很奇怪



[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: 回复:Flink SQL format问题

Jark
Administrator
文档 [1] 中写的有问题,应该是 U&'\000A'   只有一个反斜线。

Best,
Jark

[1]:
https://ci.apache.org/projects/flink/flink-docs-master/dev/table/connectors/formats/csv.html#csv-line-delimiter

On Wed, 16 Sep 2020 at 12:53, Shuai Xia <[hidden email]>
wrote:

> U&'\\000A',代码也用了U&?
>
>
> ------------------------------------------------------------------
> 发件人:[hidden email] <[hidden email]>
> 发送时间:2020年9月16日(星期三) 10:50
> 收件人:Shuai Xia <[hidden email]>
> 主 题:Re: 回复:Flink SQL format问题
>
>
>
> 表定义:
> 'connector.type' = 'kafka',
> 'connector.version' = 'universal',
> 'connector.topic' = 'xxxx',
> 'connector.properties.bootstrap.servers' = 'xxxx',
> 'connector.properties.zookeeper.connect' = 'xxxx',
> 'connector.properties.group.id' = 'xxxx',
> 'connector.properties.client.id' = 'xxxx',
> 'connector.startup-mode' = 'latest-offset',
> 'format.type' = 'csv',
> 'csv.field-delimiter' = '\001',
> 'csv.line-delimiter' = U&'\\000A'
> 错误:
> aused by: org.apache.flink.table.api.NoMatchingTableFactoryException:
> Could not find a suitable table factory for
> 'org.apache.flink.table.factories.TableSinkFactory' in
> (com.dataplatform.flink.util.FlinkDebugThread)
> [2020-09-16 10:46:52,533] INFO ---the classpath.
> (com.dataplatform.flink.util.FlinkDebugThread)
> bugThread)
> [2020-09-16 10:46:52,533] INFO ---      at
> org.apache.flink.table.planner.delegation.PlannerBase$$anonfun$1.apply(PlannerBase.scala:163)
> (com.dataplatform.flink.util.Flink
> DebugThread)
> [2020-09-16 10:46:52,533] INFO ---      at
> org.apache.flink.table.planner.delegation.PlannerBase$$anonfun$1.apply(PlannerBase.scala:163)
> (com.dataplatform.flink.util.Flink
> DebugThread)
> [2020-09-16 10:46:52,533] INFO ---      at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> (com.dataplatform.flink.util.FlinkDebugThread)
> [2020-09-16 10:46:52,533] INFO ---      at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> (com.dataplatform.flink.util.FlinkDebugThread)
> [2020-09-16 10:46:52,533] INFO ---      at
> scala.collection.Iterator$class.foreach(Iterator.scala:891)
> (com.dataplatform.flink.util.FlinkDebugThread)
> [2020-09-16 10:46:52,533] INFO ---      at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
> (com.dataplatform.flink.util.FlinkDebugThread)
> [2020-09-16 10:46:52,533] INFO ---      at
> scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> (com.dataplatform.flink.util.FlinkDebugThread)
> [2020-09-16 10:46:52,533] INFO ---      at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> (com.dataplatform.flink.util.FlinkDebugThread)
>
>
>
>
> [hidden email]
> 发件人: Shuai Xia
> 发送时间: 2020-09-16 10:33
> 收件人: user-zh; guaishushu1103
> 主题: 回复:Flink SQL format问题
> Hi,麻烦代码以及报错贴一下
>
> ------------------------------------------------------------------
> 发件人:[hidden email] <[hidden email]>
> 发送时间:2020年9月16日(星期三) 10:29
> 收件人:user-zh <[hidden email]>
> 主 题:Flink SQL format问题
>
> csv.field-delimiter
> csv.line-delimiter
> 想问下大佬们 官方文档说是可以设置这两个值,指定行分隔和字段分隔,但是在设置kafka sinkTable的时候会出现语法错误???很奇怪
>
>
>
> [hidden email]
>
Reply | Threaded
Open this post in threaded view
|

Re: Re:Flink SQL format问题

Jark
Administrator
只支持单字符。 支持不可见字符。

On Wed, 16 Sep 2020 at 15:24, [hidden email] <[hidden email]>
wrote:

> 好的 谢谢啦,csv.field-delimiter 字段分隔目前是只支持 ,和; 这种简单字符吗
>
> ------------------------------
> [hidden email]
>
>
> *发件人:* Jark Wu <[hidden email]>
> *发送时间:* 2020-09-16 15:21
> *收件人:* user-zh <[hidden email]>; Shuai Xia
> <[hidden email]>
> *抄送:* [hidden email]
> *主题:* Re: 回复:Flink SQL format问题
> 文档 [1] 中写的有问题,应该是 U&'\000A'   只有一个反斜线。
>
> Best,
> Jark
>
> [1]:
> https://ci.apache.org/projects/flink/flink-docs-master/dev/table/connectors/formats/csv.html#csv-line-delimiter
>
> On Wed, 16 Sep 2020 at 12:53, Shuai Xia <[hidden email]>
> wrote:
>
>> U&'\\000A',代码也用了U&?
>>
>>
>> ------------------------------------------------------------------
>> 发件人:[hidden email] <[hidden email]>
>> 发送时间:2020年9月16日(星期三) 10:50
>> 收件人:Shuai Xia <[hidden email]>
>> 主 题:Re: 回复:Flink SQL format问题
>>
>>
>>
>> 表定义:
>> 'connector.type' = 'kafka',
>> 'connector.version' = 'universal',
>> 'connector.topic' = 'xxxx',
>> 'connector.properties.bootstrap.servers' = 'xxxx',
>> 'connector.properties.zookeeper.connect' = 'xxxx',
>> 'connector.properties.group.id' = 'xxxx',
>> 'connector.properties.client.id' = 'xxxx',
>> 'connector.startup-mode' = 'latest-offset',
>> 'format.type' = 'csv',
>> 'csv.field-delimiter' = '\001',
>> 'csv.line-delimiter' = U&'\\000A'
>> 错误:
>> aused by: org.apache.flink.table.api.NoMatchingTableFactoryException:
>> Could not find a suitable table factory for
>> 'org.apache.flink.table.factories.TableSinkFactory' in
>> (com.dataplatform.flink.util.FlinkDebugThread)
>> [2020-09-16 10:46:52,533] INFO ---the classpath.
>> (com.dataplatform.flink.util.FlinkDebugThread)
>> bugThread)
>> [2020-09-16 10:46:52,533] INFO ---      at
>> org.apache.flink.table.planner.delegation.PlannerBase$$anonfun$1.apply(PlannerBase.scala:163)
>> (com.dataplatform.flink.util.Flink
>> DebugThread)
>> [2020-09-16 10:46:52,533] INFO ---      at
>> org.apache.flink.table.planner.delegation.PlannerBase$$anonfun$1.apply(PlannerBase.scala:163)
>> (com.dataplatform.flink.util.Flink
>> DebugThread)
>> [2020-09-16 10:46:52,533] INFO ---      at
>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
>> (com.dataplatform.flink.util.FlinkDebugThread)
>> [2020-09-16 10:46:52,533] INFO ---      at
>> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
>> (com.dataplatform.flink.util.FlinkDebugThread)
>> [2020-09-16 10:46:52,533] INFO ---      at
>> scala.collection.Iterator$class.foreach(Iterator.scala:891)
>> (com.dataplatform.flink.util.FlinkDebugThread)
>> [2020-09-16 10:46:52,533] INFO ---      at
>> scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
>> (com.dataplatform.flink.util.FlinkDebugThread)
>> [2020-09-16 10:46:52,533] INFO ---      at
>> scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>> (com.dataplatform.flink.util.FlinkDebugThread)
>> [2020-09-16 10:46:52,533] INFO ---      at
>> scala.collection.AbstractIterable.foreach(Iterable.scala:54)
>> (com.dataplatform.flink.util.FlinkDebugThread)
>>
>>
>>
>>
>> [hidden email]
>> 发件人: Shuai Xia
>> 发送时间: 2020-09-16 10:33
>> 收件人: user-zh; guaishushu1103
>> 主题: 回复:Flink SQL format问题
>> Hi,麻烦代码以及报错贴一下
>>
>> ------------------------------------------------------------------
>> 发件人:[hidden email] <[hidden email]>
>> 发送时间:2020年9月16日(星期三) 10:29
>> 收件人:user-zh <[hidden email]>
>> 主 题:Flink SQL format问题
>>
>> csv.field-delimiter
>> csv.line-delimiter
>> 想问下大佬们 官方文档说是可以设置这两个值,指定行分隔和字段分隔,但是在设置kafka sinkTable的时候会出现语法错误???很奇怪
>>
>>
>>
>> [hidden email]
>>
>