Some questions about limit push down

[Jun Zhang]

when I query hive table by sql, like this `select * from hivetable where id
= 1 limit 1`,   I found that the limit push down is invalid, is it a bug or
was it designed like this?

if the sql is  'select * from hivetable  limit 1'  ,it is ok

thanks

[Shengkai Fang]

hi, Jun Zhang.

Currently, the rule `PushLimitIntoTableSourceScanRule` captures the
structure that the `FlinkLogicalSort` node is the parent of the
`FlinkLogicalTableSourceScan`.

In your case, we have a Calc node between the Sort node and the Scan node,
which makes the rule fails to apply.

Maybe we can add a rule to transpose between the Sort and Calc.

Best,
Shengkai

Jun Zhang <[hidden email]> 于2020年12月28日周一 下午5:31写道：

> when I query hive table by sql, like this `select * from hivetable where id
> = 1 limit 1`,   I found that the limit push down is invalid, is it a bug or
> was it designed like this?
>
> if the sql is  'select * from hivetable  limit 1'  ,it is ok
>
> thanks
>

[Arvid Heise]

This is most likely a bug, could you reiterate a bit how it is invalid?
I'm also CCing Jark since he is one of the SQL experts.

On Mon, Dec 28, 2020 at 10:37 AM Jun Zhang <[hidden email]>
wrote:

> when I query hive table by sql, like this `select * from hivetable where
> id = 1 limit 1`,   I found that the limit push down is invalid, is it a bug
> or was it designed like this?
>
> if the sql is  'select * from hivetable  limit 1'  ,it is ok
>
> thanks
>


--

Arvid Heise | Senior Java Developer

<https://www.ververica.com/>

Follow us @VervericaData

--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Toni) Cheng