flink读写阿里云oss问题

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

flink读写阿里云oss问题

casel.chen
请问flink如何批/流读写阿里云oss? 我试着通过filesystem sql connector [1] 连接oss,按照官网配置oss endpoint, accessKey (id + secret)  [2] 在flink-conf.yaml文件,程序仍然报找不到 fs.oss.endpoint,我已经将 flink-oss-fs-hadoop jar放进了 plugins/目录下面。想问一下读写oss是否一定要依赖于hadoop呢?官网有提及 Hadoop Aliyun module,不知道具体做法是什么?还请给个具体实操例子,谢谢!


[1] https://ci.apache.org/projects/flink/flink-docs-master/dev/table/connectors/filesystem.html
[2] https://ci.apache.org/projects/flink/flink-docs-master/deployment/filesystems/oss.html
[3] http://hadoop.apache.org/docs/current/hadoop-aliyun/tools/hadoop-aliyun/index.html