FLINK 1.9.1 StreamingFileSink 压缩问题
Posted by CHENJIE on Jan 02, 2020; 5:36am
URL: http://apache-flink.370.s1.nabble.com/FLINK-1-9-1-StreamingFileSink-tp1369.html
各位好,FLINK 1.9.1 使用 StreamingFileSink 写Parquet到HDFS,能启用压缩吗?
--代码
StreamingFileSink<HDFSBean> sink = StreamingFileSink
.forBulkFormat(new Path(FILE_HDFS_PATH), ParquetAvroWriters.forReflectRecord(HDFSBean.class))
.withBucketAssigner(new DateTimeBucketAssigner<>(FILE_HDFS_FORMAT))
.build();