flink savepoint 异常

[张锴]

本人用flink 1.10.1版本进行savepoint时遇到下列错误，暂时不清楚错误的原因，特来寻求帮助，麻烦大佬们看看

已经排除反压和重启的原因，checkpoint超时设置了十分钟，conf配置增加客户端连接master的时间，但还是出现异常。

命令

flink savepoint -yid application_1604456903594_2381
fb8131bcb78cbdf2bb9a705d8a4ceebc
hdfs:///hadoopnamenodeHA/flink/flink-savepoints

异常

The program finished with the following exception:

org.apache.flink.util.FlinkException: Triggering a savepoint for the job
fb8131bcb78cbdf2bb9a705d8a4ceebc failed.
 at
org.apache.flink.client.cli.CliFrontend.triggerSavepoint(CliFrontend.java:631)
 at
org.apache.flink.client.cli.CliFrontend.lambda$savepoint$9(CliFrontend.java:609)
 at
org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:841)
 at org.apache.flink.client.cli.CliFrontend.savepoint(CliFrontend.java:606)
 at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:908)
 at
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:966)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
 at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
 at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:966)
Caused by: java.util.concurrent.CompletionException:
java.util.concurrent.CompletionException:
org.apache.flink.runtime.checkpoint.CheckpointException: Failed to trigger
savepoint. Failure reason: An Exception occurred while triggering the
checkpoint.
 at
org.apache.flink.runtime.scheduler.SchedulerBase.lambda$triggerSavepoint$3(SchedulerBase.java:756)
 at
java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
 at
java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797)
 at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
 at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:402)
 at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:195)
 at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
 at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
 at akka.japi.pf.UnitCaseState

[Congxian Qiu]

Hi
     异常信息中有 “Failed to trigger savepoint. Failure reason: An Exception
occurred while triggering the checkpoint.”  或许你可以看看 JM 的日志，找一下看看有没有什么详细日志
Best,
Congxian


张锴 <[hidden email]> 于2020年11月7日周六 下午4:14写道：

> 本人用flink 1.10.1版本进行savepoint时遇到下列错误，暂时不清楚错误的原因，特来寻求帮助，麻烦大佬们看看
>
> 已经排除反压和重启的原因，checkpoint超时设置了十分钟，conf配置增加客户端连接master的时间，但还是出现异常。
>
> 命令
>
> flink savepoint -yid application_1604456903594_2381
> fb8131bcb78cbdf2bb9a705d8a4ceebc
> hdfs:///hadoopnamenodeHA/flink/flink-savepoints
>
> 异常
>
> The program finished with the following exception:
>
> org.apache.flink.util.FlinkException: Triggering a savepoint for the job
> fb8131bcb78cbdf2bb9a705d8a4ceebc failed.
>  at
>
> org.apache.flink.client.cli.CliFrontend.triggerSavepoint(CliFrontend.java:631)
>  at
>
> org.apache.flink.client.cli.CliFrontend.lambda$savepoint$9(CliFrontend.java:609)
>  at
>
> org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:841)
>  at org.apache.flink.client.cli.CliFrontend.savepoint(CliFrontend.java:606)
>  at
>
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:908)
>  at
>
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:966)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at javax.security.auth.Subject.doAs(Subject.java:422)
>  at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1836)
>  at
>
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:966)
> Caused by: java.util.concurrent.CompletionException:
> java.util.concurrent.CompletionException:
> org.apache.flink.runtime.checkpoint.CheckpointException: Failed to trigger
> savepoint. Failure reason: An Exception occurred while triggering the
> checkpoint.
>  at
>
> org.apache.flink.runtime.scheduler.SchedulerBase.lambda$triggerSavepoint$3(SchedulerBase.java:756)
>  at
>
> java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:822)
>  at
>
> java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:797)
>  at
>
> java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
>  at
>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:402)
>  at
>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:195)
>  at
>
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
>  at
>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
>  at akka.japi.pf.UnitCaseState
>