Flink Weekly | 每周社区动态更新 - 2019/12/31

[zhisheng]

大家好,

很高兴与大家分享本周的社区摘要，其中包括讨论在 Flink SQL 中支持 JSON functions，新增 Flink
国内社区的活动和相关博客，以及汇总中文邮件中大家遇到的问题。



Flink 开发

=================

* [SQL] Forward Xu 发起了一个讨论，要在 Flink SQL 中支持 JSON functions，最后将讨论的结果和想法记录在了
FLIP-90
<https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=141724550>
[1]

* [SQL] Jark Wu 发起一个讨论，建议在 1.10 release 之前将 Table API 和 SQL 中的
Time-windowed Join 改为 Interval Join [2]

* [core]Stephan Ewen 之前发起的一个讨论，关于新的 TaskManager 内存配置 FLIP-49
<https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors>
[3]


[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-JSON-functions-in-Flink-SQL-td32674.html

[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Correct-the-terminology-of-quot-Time-windowed-Join-quot-to-quot-Interval-Join-quot-in-Table-L-td36202.html

[3]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Some-feedback-after-trying-out-the-new-FLIP-49-memory-configurations-td36129.html



已知缺陷

=================

[FLINK-15421 <https://issues.apache.org/jira/browse/FLINK-15421>] [1.9.1,
1.10.0] roupAggsHandler throws java.time.LocalDateTime cannot be cast to
java.sql.Timestamp [4]

[FLINK-15420 <https://issues.apache.org/jira/browse/FLINK-15420>] [1.10.0]
Cast string to timestamp will loose precision [5]

[FLINK-15418 <https://issues.apache.org/jira/browse/FLINK-15418>] [1.9.1,
1.10.0] StreamExecMatchRule not set FlinkRelDistribution [6]

[FLINK-15411 <https://issues.apache.org/jira/browse/FLINK-15411>] [1.10.0]
Planner can't prune partition on DATE/TIMESTAMP columns [7]

[FLINK-15406 <https://issues.apache.org/jira/browse/FLINK-15406>] [1.9.1]
The savepoint is writted by "State Processor API" can't be restore by map
or flatmap [8]

[FLINK-15381 <https://issues.apache.org/jira/browse/FLINK-15381>] [1.10.0]
INSERT INTO VALUES statement fails if a cast project is applied [9]

[FLINK-15370 <https://issues.apache.org/jira/browse/FLINK-15370>] [1.10.0,
1.11.0] Configured write buffer manager actually not take effect in
RocksDB's DBOptions  在 DBOptions 中设置 WriteBufferManager 的值不起作用 [10]

[FLINK-15361 <https://issues.apache.org/jira/browse/FLINK-15361>] [1.9.1]
ParquetTableSource should pass predicate in projectFields [11]

[4] https://issues.apache.org/jira/browse/FLINK-15421

[5] https://issues.apache.org/jira/browse/FLINK-15420

[6] https://issues.apache.org/jira/browse/FLINK-15418

[7] https://issues.apache.org/jira/browse/FLINK-15411

[8] https://issues.apache.org/jira/browse/FLINK-15406

[9] https://issues.apache.org/jira/browse/FLINK-15381

[10] https://issues.apache.org/jira/browse/FLINK-15370

[11] https://issues.apache.org/jira/browse/FLINK-15361



活动/博客文章/其他

=================

* 社区钉钉群群直播——《基于 Apache Flink 的监控告警系统》zhisheng

    PPT 下载地址 [12]        直播视频回放地址 [13]

* 年度回顾 | 2019 年的 Apache Flink
<https://mp.weixin.qq.com/s/0FsRGoBZaAnRYZ1BuzJ_CQ>  伍翀（云邪） [14]


[12]
https://files.alicdn.com/tpsservice/61774ae8080cd6b28145c32c99ad026a.pdf

[13] https://www.bilibili.com/video/av80497878/

[14] https://mp.weixin.qq.com/s/0FsRGoBZaAnRYZ1BuzJ_CQ



中文邮件问题/答疑汇总

=================

* StreamTableEnvironment.registerDatastream()
开放用户自定义的schemaDescriptionh和DeserializationSchema [15]

* 在 1.8 版本使用 yarn session 模式时是 1 个 TaskManager 一个 CPU，当切换到 1.9.1
时使用同样的启动命令发现一个 slot 使用一个 CPU [16]

* Flink1.9批任务yn和ys对任务的影响 [17]

* flink sql 1.9.0如何创建elasticsearch动态索引表 [18]

* source并行度不同导致任务没有数据落地 [19]

* Flink 1.9 SQL Kafka Connector，Json format，how to deal with not json
message [20]

* Flink实现Kafka到Mysql的 End-To-End Exactly-Once中遇到的问题 [21]

* Rewind offset to a previous position and ensure certainty [22]

* 关于 Flink historyserver没有completed-jobs的问题 [23]

* flink 维表关联，当维表更新后，如何将之前关联的数据进行更新 [24]

* The assigned slot was removed 分配好的slot突然就被remove了，导致作业重启 [25]

* flink checkpoint配置hdfs问题，如何配置高可用 [26]

* 实现一个两阶段提交的ETL，数据从kafka到mysql，遇到的异常问题 [27]

* using thin jar to replace fat jar on yarn cluster mode [28]

* 关于Window ProcessFunction数据丢失问题 [29]

* 关于flink窗口是否正确关闭的问题 [30]

* CEP匹配乱序数据的问题 [31]


[15]
http://apache-flink.147419.n8.nabble.com/StreamTableEnvironment-registerDatastream-schemaDescriptionh-DeserializationSchema-tp1347.html

[16] http://apache-flink.147419.n8.nabble.com/slot-td1345.html

[17] http://apache-flink.147419.n8.nabble.com/Flink1-9-yn-ys-tp1313.html

[18]
http://apache-flink.147419.n8.nabble.com/flink-sql-1-9-0-elasticsearch-tp1342.html

[19] http://apache-flink.147419.n8.nabble.com/source-tp1336.html

[20]
http://apache-flink.147419.n8.nabble.com/Flink-1-9-SQL-Kafka-Connector-Json-format-how-to-deal-with-not-json-message-tp1335.html

[21]
http://apache-flink.147419.n8.nabble.com/Flink-Kafka-Mysql-End-To-End-Exactly-Once-tp1321.html

[22]
http://apache-flink.147419.n8.nabble.com/Rewind-offset-to-a-previous-position-and-ensure-certainty-tp1324.html

[23]
http://apache-flink.147419.n8.nabble.com/FLink-historyserver-completed-jobs-tp1320.html

[24] http://apache-flink.147419.n8.nabble.com/flink-tp1322.html

[25]
http://apache-flink.147419.n8.nabble.com/The-assigned-slot-bae00218c818157649eb9e3c533b86af-11-was-removed-td1308.html

[26]
http://apache-flink.147419.n8.nabble.com/flink-checkpoint-hdfs-tp1318.html

[27] http://apache-flink.147419.n8.nabble.com/ETL-kafka-mysql-tp1317.html

[28]
http://apache-flink.147419.n8.nabble.com/using-thin-jar-to-replace-fat-jar-on-yarn-cluster-mode-tp1293.html

[29]
http://apache-flink.147419.n8.nabble.com/Window-ProcessFunction-tp1311.html

[30] http://apache-flink.147419.n8.nabble.com/flink-tp1305.html

[31] http://apache-flink.147419.n8.nabble.com/CEP-tp1301.html

谢谢,

zhisheng