关于使用Flink RocksDBStateBackend问题

[chanamper]

Hi
在使用RocksDBStateBackend过程中，有些问题想请教一下
采用自定义的optionsFactory后，flink-conf.yaml配置文件中的RocksDB Configurable Options相关参数配置不生效，请问一下要如何使其中的state.backend.rocksdb.writebuffer.size等参数生效？

OptionsFactory optionsFactory = new OptionsFactory() {

    @Override

    public DBOptions createDBOptions(DBOptions dbOptions) {

     return (new DBOptions()).setUseFsync(false);

    }




    @Override

    public ColumnFamilyOptions createColumnOptions(ColumnFamilyOptions columnFamilyOptions) {

     return (new ColumnFamilyOptions()).setCompressionType(CompressionType.NO_COMPRESSION);

    }

   };

   RocksDBStateBackend rocksDBStateBackend = new RocksDBStateBackend("hdfs://:/flink/checkpoints");

   rocksDBStateBackend.setOptions(optionsFactory);

   env.setStateBackend(rocksDBStateBackend);

[Yun Tang]

Hi

使用自定义options factory的话，我们会认为是高级用户，自然也就完全交由用户进行配置，至于write buffer size如何配置，可以参考PredefinedOptions [1] 的使用方法。

[1] https://github.com/apache/flink/blob/8f67d1d7e6809d528fe957cb4eb78308d87da324/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/PredefinedOptions.java#L173

祝好
唐云

________________________________
From: chanamper <[hidden email]>
Sent: Tuesday, January 14, 2020 22:32
To: user-zh <[hidden email]>
Subject: 关于使用Flink RocksDBStateBackend问题

Hi
在使用RocksDBStateBackend过程中，有些问题想请教一下
采用自定义的optionsFactory后，flink-conf.yaml配置文件中的RocksDB Configurable Options相关参数配置不生效，请问一下要如何使其中的state.backend.rocksdb.writebuffer.size等参数生效？

OptionsFactory optionsFactory = new OptionsFactory() {

    @Override

    public DBOptions createDBOptions(DBOptions dbOptions) {

     return (new DBOptions()).setUseFsync(false);

    }




    @Override

    public ColumnFamilyOptions createColumnOptions(ColumnFamilyOptions columnFamilyOptions) {

     return (new ColumnFamilyOptions()).setCompressionType(CompressionType.NO_COMPRESSION);

    }

   };

   RocksDBStateBackend rocksDBStateBackend = new RocksDBStateBackend("hdfs://:/flink/checkpoints");

   rocksDBStateBackend.setOptions(optionsFactory);

   env.setStateBackend(rocksDBStateBackend);