如果输入有回撤流的话,group by时间窗口会遇到GroupWindowAggregate doesn't support consuming update and delete changes

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

如果输入有回撤流的话,group by时间窗口会遇到GroupWindowAggregate doesn't support consuming update and delete changes

HongHuangNeu
如果输入有回撤流的话,group by时间窗口会遇到GroupWindowAggregate doesn't support consuming
update and delete changes,有没有什么替代方案?输入是来自于流式去重,就是

SELECT [column_list]
FROM (
   SELECT [column_list],
     ROW_NUMBER() OVER ([PARTITION BY col1[, col2...]]
       ORDER BY time_attr [asc|desc]) AS rownum
   FROM table_name)
WHERE rownum = 1

这样的语句



--
Sent from: http://apache-flink.147419.n8.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: 如果输入有回撤流的话,group by时间窗口会遇到GroupWindowAggregate doesn't support consuming update and delete changes

nick
flink window  doesn't support update stream.

HongHuangNeu <[hidden email]> 于2021年2月4日周四 上午9:24写道:

> 如果输入有回撤流的话,group by时间窗口会遇到GroupWindowAggregate doesn't support consuming
> update and delete changes,有没有什么替代方案?输入是来自于流式去重,就是
>
> SELECT [column_list]
> FROM (
>    SELECT [column_list],
>      ROW_NUMBER() OVER ([PARTITION BY col1[, col2...]]
>        ORDER BY time_attr [asc|desc]) AS rownum
>    FROM table_name)
> WHERE rownum = 1
>
> 这样的语句
>
>
>
> --
> Sent from: http://apache-flink.147419.n8.nabble.com/
>
Reply | Threaded
Open this post in threaded view
|

Re:Re: 如果输入有回撤流的话,group by时间窗口会遇到GroupWindowAggregate doesn't support consuming update and delete changes

casel.chen
flink window doesn't support retract stream 的话有什么workaround办法吗?常见的场景有 业务表cdc -> kakfa -> flink按时间窗口聚合
如果业务表是只会insert的日志表,该如何将retract table转换成普通table?




GroupWindowAggregate doesn't support consuming update and delete changes which is produced by node TableSourceScan(table=[[default_catalog, default_database, mcsp_pay_log, ...

按时间窗口聚合不支持上游是canal-json格式的cdc表的情况么?我的业务表其实是一张日志表,怎样用flink sql将retract table转成append table?














在 2021-02-04 09:26:30,"yang nick" <[hidden email]> 写道:

>flink window  doesn't support update stream.
>
>HongHuangNeu <[hidden email]> 于2021年2月4日周四 上午9:24写道:
>
>> 如果输入有回撤流的话,group by时间窗口会遇到GroupWindowAggregate doesn't support consuming
>> update and delete changes,有没有什么替代方案?输入是来自于流式去重,就是
>>
>> SELECT [column_list]
>> FROM (
>>    SELECT [column_list],
>>      ROW_NUMBER() OVER ([PARTITION BY col1[, col2...]]
>>        ORDER BY time_attr [asc|desc]) AS rownum
>>    FROM table_name)
>> WHERE rownum = 1
>>
>> 这样的语句
>>
>>
>>
>> --
>> Sent from: http://apache-flink.147419.n8.nabble.com/
>>