hi各位,flink 1.11.1写s3的时候输出以下日志
2020-09-11 11:06:14,680 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=1 (max part counter=2). 2020-09-11 11:06:14,685 WARN com.amazonaws.services.s3.internal.Mimetypes [] - Unable to find 'mime.types' file in classpath 2020-09-11 11:06:15,133 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=1. 2020-09-11 11:06:15,138 INFO org.apache.flink.fs.s3.common.writer.S3Committer [] - Committing test.txt/2020-09-11--11/part-0-0 with MPU ID 2~dD7KNpNcq4Tr95ipKxT6PSEi21Yz3oe 2020-09-11 11:07:15,143 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=2 (max part counter=2). 2020-09-11 11:07:15,447 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=2. 2020-09-11 11:08:15,463 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=3 (max part counter=2). 2020-09-11 11:08:15,651 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=3. 2020-09-11 11:09:15,671 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=4 (max part counter=2). 2020-09-11 11:09:15,994 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=4. 2020-09-11 11:10:16,078 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=5 (max part counter=2). 2020-09-11 11:10:16,304 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=5. 2020-09-11 11:11:16,317 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=6 (max part counter=2). 2020-09-11 11:11:16,497 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=6. 2020-09-11 11:12:16,508 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=7 (max part counter=2). 2020-09-11 11:12:16,643 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=7. 2020-09-11 11:13:16,665 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=8 (max part counter=2). 2020-09-11 11:13:17,031 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=8. 2020-09-11 11:14:17,045 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 checkpointing for checkpoint with id=9 (max part counter=2). 2020-09-11 11:14:17,225 INFO org.apache.flink.streaming.api.functions.sink.filesystem.Buckets [] - Subtask 0 received completion notification for checkpoint with id=9. 显示一直在commiting,下游看不到任何文件输出,请问写s3的时候是先写在本地缓存的吗?什么条件下才会在下游s3中生成文件? |
Free forum by Nabble | Edit this page |