Apache Flink 中文用户邮件列表

Flink webui 查询任务信息报错500

Classic

List

Threaded

5 messages Options

nobleyd

Flink webui 查询任务信息报错500

报错500，开发者工具展示的异常信息如下。
<Exception on server
side:↵org.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
The method requestJob's result size 19811407 exceeds the maximum size
10485760 .↵ at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:363)↵
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$0(AkkaRpcActor.java:337)↵
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)↵
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)↵
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)↵
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)↵
at
org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:229)↵
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)↵
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)↵
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)↵
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)↵
at
org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:996)↵
at akka.dispatch.OnComplete.internal(Future.scala:264)↵ at
akka.dispatch.OnComplete.internal(Future.scala:261)↵ at
akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)↵ at
akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)↵ at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)↵ at
org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:74)↵
at
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)↵
at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)↵
at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)↵ at
akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)↵
at
akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)↵
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)↵ at
scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)↵ at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)↵ at
akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)↵
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)↵
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)↵ at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)↵
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)↵ at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)↵
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)↵
at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)↵↵End
of exception on server side>"

想知道这个是什么情况，以及需要调整什么参数呢？

Evan

Re: Flink webui 查询任务信息报错500

这是flink的Akka部分报的错，相关源码如下，可以找找这个 maximumFramesize 怎么修改？

https://github.com/apache/flink/blob/d093611b5dfab95fe62e4f861879762ca2e43437/flink-runtime/src/main/java/org/apache/flink/runtime/rpc/akka/AkkaRpcActor.java：

private Either<SerializedValue<?>, AkkaRpcException> serializeRemoteResultAndVerifySize(
Object result, String methodName) {
try {
SerializedValue<?> serializedResult = new SerializedValue<>(result);

long resultSize =
serializedResult.getByteArray() == null
? 0
: serializedResult.getByteArray().length;
if (resultSize > maximumFramesize) {
return Either.Right(
new AkkaRpcException(
"The method "
+ methodName
+ "'s result size "
+ resultSize
+ " exceeds the maximum size "
+ maximumFramesize
+ " ."));
} else {
return Either.Left(serializedResult);
}
} catch (IOException e) {
return Either.Right(
new AkkaRpcException(
"Failed to serialize the result for RPC call : " + methodName + '.',
e));
}
}

发件人：赵一旦
发送时间： 2021-01-14 11:38
收件人： user-zh
主题： Flink webui 查询任务信息报错500
报错500，开发者工具展示的异常信息如下。
<Exception on server
side:↵org.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
The method requestJob's result size 19811407 exceeds the maximum size
10485760 .↵ at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:363)↵
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$0(AkkaRpcActor.java:337)↵
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)↵
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)↵
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)↵
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)↵
at
org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:229)↵
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)↵
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)↵
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)↵
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)↵
at
org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:996)↵
at akka.dispatch.OnComplete.internal(Future.scala:264)↵ at
akka.dispatch.OnComplete.internal(Future.scala:261)↵ at
akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)↵ at
akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)↵ at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)↵ at
org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:74)↵
at
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)↵
at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)↵
at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)↵ at
akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)↵
at
akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)↵
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)↵ at
scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)↵ at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)↵ at
akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)↵
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)↵
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)↵ at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)↵
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)↵ at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)↵
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)↵
at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)↵↵End
of exception on server side>"

想知道这个是什么情况，以及需要调整什么参数呢？

Evan

Re: Flink webui 查询任务信息报错500

In reply to this post by nobleyd

有这样一个参数“akka.framesize” ，可以在你启动flink的时候加上或者在conf/flink-conf.yaml 配置上：

akka.framesize
"10485760b"StringMaximum size of messages which are sent between the JobManager and the TaskManagers. If Flink fails because messages exceed this limit, then you should increase it. The message size requires a size-unit specifier.

参考：
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/config.html

发件人：赵一旦
发送时间： 2021-01-14 11:38
收件人： user-zh
主题： Flink webui 查询任务信息报错500
报错500，开发者工具展示的异常信息如下。
<Exception on server
side:↵org.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
The method requestJob's result size 19811407 exceeds the maximum size
10485760 .↵ at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:363)↵
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$0(AkkaRpcActor.java:337)↵
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)↵
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)↵
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)↵
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)↵
at
org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:229)↵
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)↵
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)↵
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)↵
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)↵
at
org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:996)↵
at akka.dispatch.OnComplete.internal(Future.scala:264)↵ at
akka.dispatch.OnComplete.internal(Future.scala:261)↵ at
akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)↵ at
akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)↵ at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)↵ at
org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:74)↵
at
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)↵
at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)↵
at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)↵ at
akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)↵
at
akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)↵
at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)↵ at
scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)↵ at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)↵ at
akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)↵
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)↵
at
akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)↵
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)↵ at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)↵
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)↵ at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)↵
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)↵
at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)↵↵End
of exception on server side>"

想知道这个是什么情况，以及需要调整什么参数呢？

nobleyd

Re: Flink webui 查询任务信息报错500

In reply to this post by nobleyd

好的，我找到了这个参数。不过这个参数表达啥含义知道吗，我看10MB不是个小数字感觉。

Evan <[hidden email]> 于2021年1月14日周四下午1:54写道：

> 有这样一个参数“akka.framesize” ，可以在你启动flink的时候加上或者在conf/flink-conf.yaml 配置上：
>
> akka.framesize
> "10485760b"StringMaximum size of messages which are sent between the
> JobManager and the TaskManagers. If Flink fails because messages exceed
> this limit, then you should increase it. The message size requires a
> size-unit specifier.
>
> 参考：
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/config.html
>
>
>
>
>
> 发件人：赵一旦
> 发送时间： 2021-01-14 11:38
> 收件人： user-zh
> 主题： Flink webui 查询任务信息报错500
> 报错500，开发者工具展示的异常信息如下。
> <Exception on server
> side:↵org.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
> The method requestJob's result size 19811407 exceeds the maximum size
> 10485760 .↵ at
>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:363)↵
> at
>
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$0(AkkaRpcActor.java:337)↵
> at
>
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)↵
> at
>
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)↵
> at
>
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)↵
> at
>
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)↵
> at
>
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:229)↵
> at
>
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)↵
> at
>
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)↵
> at
>
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)↵
> at
>
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)↵
> at
>
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:996)↵
> at akka.dispatch.OnComplete.internal(Future.scala:264)↵ at
> akka.dispatch.OnComplete.internal(Future.scala:261)↵ at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)↵ at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)↵ at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)↵ at
>
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:74)↵
> at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)↵
> at
>
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)↵
> at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)↵ at
>
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)↵
> at
>
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)↵
> at scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)↵ at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)↵ at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)↵ at
>
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)↵
> at
>
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)↵
> at
>
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)↵
> at
>
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)↵
> at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)↵
> at
>
> akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)↵
> at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)↵ at
>
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)↵
> at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)↵ at
>
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)↵
> at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)↵
> at
>
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)↵↵End
> of exception on server side>"
>
> 想知道这个是什么情况，以及需要调整什么参数呢？
>

Evan

Re: Re: Flink webui 查询任务信息报错500

你可以从akka的原理架构中获取一些信息
参考： https://akka.io

之前在其他博主的介绍中看到过，感觉意思差不多，上面说这是JobManager和TaskManager之间通信时发送的消息大小的最大值

发件人：赵一旦
发送时间： 2021-01-14 14:19
收件人： user-zh
主题： Re: Flink webui 查询任务信息报错500
好的，我找到了这个参数。不过这个参数表达啥含义知道吗，我看10MB不是个小数字感觉。

Evan <[hidden email]> 于2021年1月14日周四下午1:54写道：