周期性batch mode 从 hive 提取数据插入 mysql,每批次 10K 到 20K 行数据,多数情况下
10-20秒可以完成,但不定期就会很长时间,能达到 20多分钟,但也能成功,查看了日志也看不到错误,检查 mysql 也没有发现锁表,怀疑 hive metastore 的性能,但也没看出问题。 请教分析思路,从 flink 上能看出job 在等待什么吗? -- Sent from: http://apache-flink.147419.n8.nabble.com/ |
打开了 debug 级别的日志,有这样的错误
2021-01-31 20:45:30,364 DEBUG org.apache.flink.runtime.io.network.partition.ResultPartitionManager [] - Released partition dc8a2804b6df6b0ceaee2610ccf6c6e5#312 produced by 448c5ac36dcda818f56ec5bbd728da10. 2021-01-31 20:45:30,392 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 80a1592c9e59efd80e412e7dee99f70c because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,392 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:2, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 80a1592c9e59efd80e412e7dee99f70c, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,393 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 80a1592c9e59efd80e412e7dee99f70c. 2021-01-31 20:45:30,393 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 4606a6194b4380efb5c2f95fc65bf01e because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,393 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:12, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 4606a6194b4380efb5c2f95fc65bf01e, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,393 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 4606a6194b4380efb5c2f95fc65bf01e. 2021-01-31 20:45:30,393 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 00a8fb496d02878a419b4cb162714f75 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,393 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:8, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 00a8fb496d02878a419b4cb162714f75, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,394 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 00a8fb496d02878a419b4cb162714f75. 2021-01-31 20:45:30,394 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 1cc4d7e3ec58f94fc3e3c9cca15d1525 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,394 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:23, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 1cc4d7e3ec58f94fc3e3c9cca15d1525, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,394 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 1cc4d7e3ec58f94fc3e3c9cca15d1525. 2021-01-31 20:45:30,395 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 78cfae8070fa5f09f0195111290c114e because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,395 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:18, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 78cfae8070fa5f09f0195111290c114e, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,395 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 78cfae8070fa5f09f0195111290c114e. 2021-01-31 20:45:30,395 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 91cb1d238266a31706cf5bf1e1feb061 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,395 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:30, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 91cb1d238266a31706cf5bf1e1feb061, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,395 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 91cb1d238266a31706cf5bf1e1feb061. 2021-01-31 20:45:30,396 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id e24a2b93f01205a074fba9d80043610c because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,396 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:5, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: e24a2b93f01205a074fba9d80043610c, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,396 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id e24a2b93f01205a074fba9d80043610c. 2021-01-31 20:45:30,396 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 3de4aa1ac1d479f31cea927924bb75e5 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,396 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:15, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 3de4aa1ac1d479f31cea927924bb75e5, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,396 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 3de4aa1ac1d479f31cea927924bb75e5. 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id e658a14fb8402e922305dc8e881c963c because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:3, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: e658a14fb8402e922305dc8e881c963c, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id e658a14fb8402e922305dc8e881c963c. 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 9ad7f0b35dc6814d5485f0616cd6cfc7 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:10, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 9ad7f0b35dc6814d5485f0616cd6cfc7, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 9ad7f0b35dc6814d5485f0616cd6cfc7. 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id a499129a67f9a7dee8387071ad2c0338 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:27, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: a499129a67f9a7dee8387071ad2c0338, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,397 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id a499129a67f9a7dee8387071ad2c0338. 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 2b60e444c6103c8c95c5825d9e01cc40 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:1, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 2b60e444c6103c8c95c5825d9e01cc40, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 2b60e444c6103c8c95c5825d9e01cc40. 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id ad9dc473be1c7fdbe603a21d957a3902 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:28, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: ad9dc473be1c7fdbe603a21d957a3902, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id ad9dc473be1c7fdbe603a21d957a3902. 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 98a071f51297851d0b6ce6d6271fd4d0 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:7, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 98a071f51297851d0b6ce6d6271fd4d0, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,398 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 98a071f51297851d0b6ce6d6271fd4d0. 2021-01-31 20:45:30,399 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 889f5bc55a1553762c7751245f772944 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,399 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:0, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 889f5bc55a1553762c7751245f772944, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,399 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 889f5bc55a1553762c7751245f772944. 2021-01-31 20:45:30,399 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 2915d09a6898fd59fa1de068421ca422 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,399 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:9, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 2915d09a6898fd59fa1de068421ca422, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,399 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 2915d09a6898fd59fa1de068421ca422. 2021-01-31 20:45:30,399 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 284e3e67adb45f654452a6190b921c70 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,399 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:14, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 284e3e67adb45f654452a6190b921c70, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,400 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 284e3e67adb45f654452a6190b921c70. 2021-01-31 20:45:30,400 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id d8538ef2f2f0047463826a0d19cf91a9 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,400 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:4, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: d8538ef2f2f0047463826a0d19cf91a9, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,400 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id d8538ef2f2f0047463826a0d19cf91a9. 2021-01-31 20:45:30,400 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 1908db91886c67a17c92691b4f076798 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,400 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:17, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 1908db91886c67a17c92691b4f076798, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,400 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 1908db91886c67a17c92691b4f076798. 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id ac7bcd13f553b941397815e1346595a4 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:26, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: ac7bcd13f553b941397815e1346595a4, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id ac7bcd13f553b941397815e1346595a4. 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id b51d5e199e94f6b5b73d750992f083b9 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:11, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: b51d5e199e94f6b5b73d750992f083b9, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id b51d5e199e94f6b5b73d750992f083b9. 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 3af5df93ddecf04ed19a4bcbba192843 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:29, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 3af5df93ddecf04ed19a4bcbba192843, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,401 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 3af5df93ddecf04ed19a4bcbba192843. 2021-01-31 20:45:30,402 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id baa76eafbe6960b293b9052619844550 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,402 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:20, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: baa76eafbe6960b293b9052619844550, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,402 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id baa76eafbe6960b293b9052619844550. 2021-01-31 20:45:30,402 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 3b7554cf7b819ea02fcd7510448db423 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,402 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:25, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 3b7554cf7b819ea02fcd7510448db423, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,402 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 3b7554cf7b819ea02fcd7510448db423. 2021-01-31 20:45:30,402 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 33e8b890a1b7db73e13256c7b857fe19 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,402 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:6, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 33e8b890a1b7db73e13256c7b857fe19, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,403 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 33e8b890a1b7db73e13256c7b857fe19. 2021-01-31 20:45:30,403 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 37feb60c758c477a296c60b6d28eb92f because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,403 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:13, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 37feb60c758c477a296c60b6d28eb92f, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,403 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 37feb60c758c477a296c60b6d28eb92f. 2021-01-31 20:45:30,403 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id f4c44e0ccc3f1f6dcb0620ba5a6109fd because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,403 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:16, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: f4c44e0ccc3f1f6dcb0620ba5a6109fd, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,403 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id f4c44e0ccc3f1f6dcb0620ba5a6109fd. 2021-01-31 20:45:30,403 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id ce29214078349bd40c06ce5735821a5c because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,404 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:31, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: ce29214078349bd40c06ce5735821a5c, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,404 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id ce29214078349bd40c06ce5735821a5c. 2021-01-31 20:45:30,404 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id 027ea2c6b31426d481d57e2de997a351 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,404 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:19, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: 027ea2c6b31426d481d57e2de997a351, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,404 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id 027ea2c6b31426d481d57e2de997a351. 2021-01-31 20:45:30,404 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Free slot with allocation id aa12b9511f6570980da62e7a284e3775 because: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). 2021-01-31 20:45:30,404 DEBUG org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:24, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=344.000mb (360710140 bytes), taskOffHeapMemory=0 bytes, managedMemory=256.000mb (268435460 bytes), networkMemory=32.000mb (33554432 bytes)}, allocationId: aa12b9511f6570980da62e7a284e3775, jobId: d055754b88483b13648cc3fb32d9cd58). org.apache.flink.util.FlinkException: Stopping JobMaster for job ifXTable->p_port_traffic_5m @2021-01-31 20:30:00(d055754b88483b13648cc3fb32d9cd58). at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:416) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:565) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:187) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[flink-dist_2.11-1.12.1.jar:1.12.1] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.11-1.12.1.jar:1.12.1] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.11-1.12.1.jar:1.12.1] 2021-01-31 20:45:30,407 DEBUG org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Releasing local state under allocation id aa12b9511f6570980da62e7a284e3775. 2021-01-31 20:45:30,782 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Received heartbeat request from 88aa75d8ed330279fee0a9eace377083. 2021-01-31 20:45:34,985 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Got ping response for sessionid: 0x40062446332000c after 0ms 2021-01-31 20:45:36,262 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Received heartbeat request from 05ce1815462d39411d689a63828ccc03. 2021-01-31 20:45:41,932 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Received heartbeat request from 179f60ac95a20681fe621d2a50e78924. 2021-01-31 20:45:48,331 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Got ping response for sessionid: 0x40062446332000c after 1ms 2021-01-31 20:45:50,802 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Received heartbeat request from 88aa75d8ed330279fee0a9eace377083. 2021-01-31 20:45:56,282 DEBUG org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Received heartbeat request from 05ce1815462d39411d689a63828ccc03. 2021-01-31 20:46:01,678 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Got ping response for sessionid: 0x40062446332000c after 1ms 2021-01-31 20:46:01,923 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Receive slot request bdbb6bc30646e4e7e6e58907bc87bc23 for job e20992b9bdbc0637b6c68350bbf4763a from resource manager with leader id a3443ae616317119ecea3ff8b3cb4c1a. 2021-01-31 20:46:01,923 DEBUG org.apache.flink.runtime.memory.MemoryManager [] - Initialized MemoryManager with total memory size 268435460 and page size 32768. 2021-01-31 20:46:01,923 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Allocated slot for bdbb6bc30646e4e7e6e58907bc87bc23. 2021-01-31 20:46:01,923 INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Add job e20992b9bdbc0637b6c68350bbf4763a for job leader monitoring. 2021-01-31 20:46:01,923 INFO org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - Starting DefaultLeaderRetrievalService with ZookeeperLeaderRetrievalDriver{retrievalPath='/leader/e20992b9bdbc0637b6c68350bbf4763a/job_manager_lock'}. 2021-01-31 20:46:01,924 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Receive slot request 573c2cf13dd8474dca2a37ce45ed7bc3 for job e20992b9bdbc0637b6c68350bbf4763a from resource manager with leader id a3443ae616317119ecea3ff8b3cb4c1a. 2021-01-31 20:46:01,924 DEBUG org.apache.flink.runtime.memory.MemoryManager [] - Initialized MemoryManager with total memory size 268435460 and page size 32768. 2021-01-31 20:46:01,924 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Allocated slot for 573c2cf13dd8474dca2a37ce45ed7bc3. 2021-01-31 20:46:01,924 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Reading reply sessionid:0x40062446332000c, packet:: clientPath:null serverPath:null finished:false header:: 457,3 replyHeader:: 457,60144709024,0 request:: '/flink,F response:: s{60144679485,60144679485,1612072709179,1612072709179,0,1,0,0,0,1,60144679487} 2021-01-31 20:46:01,925 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Reading reply sessionid:0x40062446332000c, packet:: clientPath:null serverPath:null finished:false header:: 458,3 replyHeader:: 458,60144709024,0 request:: '/flink/cluster_flink,F response:: s{60144679487,60144679487,1612072709183,1612072709183,0,6,0,0,0,6,60144679527} 2021-01-31 20:46:01,925 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Receive slot request dc41edc20a4ee3c8f226f5340bd23936 for job e20992b9bdbc0637b6c68350bbf4763a from resource manager with leader id a3443ae616317119ecea3ff8b3cb4c1a. 2021-01-31 20:46:01,925 DEBUG org.apache.flink.runtime.memory.MemoryManager [] - Initialized MemoryManager with total memory size 268435460 and page size 32768. 2021-01-31 20:46:01,925 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Allocated slot for dc41edc20a4ee3c8f226f5340bd23936. 2021-01-31 20:46:01,925 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Reading reply sessionid:0x40062446332000c, packet:: clientPath:null serverPath:null finished:false header:: 459,3 replyHeader:: 459,60144709024,0 request:: '/flink/cluster_flink/leader,F response:: s{60144679489,60144679489,1612072709189,1612072709189,0,712,0,0,0,700,60144709016} 2021-01-31 20:46:01,925 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Reading reply sessionid:0x40062446332000c, packet:: clientPath:null serverPath:null finished:false header:: 460,3 replyHeader:: 460,60144709024,0 request:: '/flink/cluster_flink/leader/e20992b9bdbc0637b6c68350bbf4763a,F response:: s{60144709016,60144709016,1612097161651,1612097161651,0,1,0,0,0,1,60144709024} 2021-01-31 20:46:01,926 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Reading reply sessionid:0x40062446332000c, packet:: clientPath:/flink/cluster_flink/leader/e20992b9bdbc0637b6c68350bbf4763a/job_manager_lock serverPath:/flink/cluster_flink/leader/e20992b9bdbc0637b6c68350bbf4763a/job_manager_lock finished:false header:: 461,3 replyHeader:: 461,60144709024,0 request:: '/flink/cluster_flink/leader/e20992b9bdbc0637b6c68350bbf4763a/job_manager_lock,T response:: s{60144709024,60144709024,1612097161917,1612097161917,0,0,0,432362652266397718,154,0,60144709024} 2021-01-31 20:46:01,926 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Receive slot request ebc91f9d41b4f6d9e2c06096c841e04e for job e20992b9bdbc0637b6c68350bbf4763a from resource manager with leader id a3443ae616317119ecea3ff8b3cb4c1a. 2021-01-31 20:46:01,926 DEBUG org.apache.flink.runtime.memory.MemoryManager [] - Initialized MemoryManager with total memory size 268435460 and page size 32768. 2021-01-31 20:46:01,926 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Allocated slot for ebc91f9d41b4f6d9e2c06096c841e04e. 2021-01-31 20:46:01,927 DEBUG org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Reading reply sessionid:0x40062446332000c, packet:: clientPath:/flink/cluster_flink/leader/e20992b9bdbc0637b6c68350bbf4763a/job_manager_lock serverPath:/flink/cluster_flink/leader/e20992b9bdbc0637b6c68350bbf4763a/job_manager_lock finished:false header:: 462,4 replyHeader:: 462,60144709024,0 request:: '/flink/cluster_flink/leader/e20992b9bdbc0637b6c68350bbf4763a/job_manager_lock,T response:: #ffffffacffffffed057748046616b6b612e7463703a2f2f666c696e6b406862332d70726f642d6861646f6f706e616d652d3030303a34363633372f757365722f7270632f6a6f626d616e616765725f35323273720e6a6176612e7574696c2e55554944ffffffbcffffff993fffffff7ffffff986dffffff852f2024a0c6c65617374536967426974734a0b6d6f7374536967426974737870ffffff8e347d5421ffffffd8ffffff9c63ffffff9dffffffb56e12ffffffa2f43f,s{60144709024,60144709024,1612097161917,1612097161917,0,0,0,432362652266397718,154,0,60144709024} 2021-01-31 20:46:01,927 DEBUG org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalDriver [] - Leader node has changed. 2021-01-31 20:46:01,927 DEBUG org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - New leader information: Leader=akka.tcp://flink@hb3-prod-hadoopname-000:46637/user/rpc/jobmanager_522, session ID=9db56e12-a20f-4 -- Sent from: http://apache-flink.147419.n8.nabble.com/ |
In reply to this post by macdoor
有时候这种job持续2个多小时,我只能cancel job,但无法正常 cancel,都会导致 taskmanager 挂掉,错误如下
2021-01-31 23:04:23,677 ERROR org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Task did not exit gracefully within 180 + seconds. org.apache.flink.util.FlinkRuntimeException: Task did not exit gracefully within 180 + seconds. at org.apache.flink.runtime.taskmanager.Task$TaskCancelerWatchDog.run(Task.java:1685) [flink-dist_2.11-1.12.1.jar:1.12.1] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282] 2021-01-31 23:04:23,685 ERROR org.apache.flink.runtime.taskexecutor.TaskManagerRunner [] - Fatal error occurred while executing the TaskManager. Shutting it down... org.apache.flink.util.FlinkRuntimeException: Task did not exit gracefully within 180 + seconds. at org.apache.flink.runtime.taskmanager.Task$TaskCancelerWatchDog.run(Task.java:1685) [flink-dist_2.11-1.12.1.jar:1.12.1] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282] 2021-01-31 23:04:23,686 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Stopping TaskExecutor akka.tcp://flink@10.13.69.52:45901/user/rpc/taskmanager_0. 2021-01-31 23:04:23,686 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Close ResourceManager connection 1bd159f361d86e77d17e261ab44b5128. 2021-01-31 23:04:23,689 WARN org.apache.flink.runtime.taskmanager.Task [] - Task 'Source: HiveSource-snmpprobe.p_port_traffic_5m -> Calc(select=[binaryid AS id, ver, CAST(2021-01-31 21:45:00:TIMESTAMP(6)) AS coltime, CAST(in_octets) AS in_octets, CAST(out_octets) AS out_octets, CAST(bi_octets) AS bi_octets, CAST(unimax_octets) AS unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, CAST((() DATE_FORMAT _UTF-16LE'yyyy-MM-dd HH:mm:ss')) AS crtime], where=[(coltime = 2021-01-31 21:45:00:TIMESTAMP(9))]) -> Sink: Sink(table=[myhive.prod_mysql_zqzynetdb.p_port_traffic_5m], fields=[id, ver, coltime, in_octets, out_octets, bi_octets, unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, crtime]) (1/1)#0' did not react to cancelling signal for 30 seconds, but is stuck in method: java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.socketRead(SocketInputStream.java:116) java.net.SocketInputStream.read(SocketInputStream.java:171) java.net.SocketInputStream.read(SocketInputStream.java:141) com.mysql.cj.protocol.ReadAheadInputStream.fill(ReadAheadInputStream.java:107) com.mysql.cj.protocol.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:150) com.mysql.cj.protocol.ReadAheadInputStream.read(ReadAheadInputStream.java:180) java.io.FilterInputStream.read(FilterInputStream.java:133) com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:64) com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:63) com.mysql.cj.protocol.a.SimplePacketReader.readHeader(SimplePacketReader.java:45) com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:52) com.mysql.cj.protocol.a.TimeTrackingPacketReader.readHeader(TimeTrackingPacketReader.java:41) com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:54) com.mysql.cj.protocol.a.MultiPacketReader.readHeader(MultiPacketReader.java:44) com.mysql.cj.protocol.a.NativeProtocol.readMessage(NativeProtocol.java:538) com.mysql.cj.protocol.a.NativeProtocol.checkErrorMessage(NativeProtocol.java:708) com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:647) com.mysql.cj.protocol.a.NativeProtocol.sendQueryPacket(NativeProtocol.java:946) com.mysql.cj.NativeSession.execSQL(NativeSession.java:1075) com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:930) com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1092) com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchSerially(ClientPreparedStatement.java:832) com.mysql.cj.jdbc.ClientPreparedStatement.executeBatchInternal(ClientPreparedStatement.java:435) com.mysql.cj.jdbc.StatementImpl.executeBatch(StatementImpl.java:796) org.apache.flink.connector.jdbc.statement.FieldNamedPreparedStatementImpl.executeBatch(FieldNamedPreparedStatementImpl.java:65) org.apache.flink.connector.jdbc.internal.executor.TableSimpleStatementExecutor.executeBatch(TableSimpleStatementExecutor.java:64) org.apache.flink.connector.jdbc.internal.executor.TableBufferReducedStatementExecutor.executeBatch(TableBufferReducedStatementExecutor.java:101) org.apache.flink.connector.jdbc.internal.JdbcBatchingOutputFormat.attemptFlush(JdbcBatchingOutputFormat.java:216) org.apache.flink.connector.jdbc.internal.JdbcBatchingOutputFormat.flush(JdbcBatchingOutputFormat.java:184) org.apache.flink.connector.jdbc.internal.JdbcBatchingOutputFormat.writeRecord(JdbcBatchingOutputFormat.java:167) org.apache.flink.streaming.api.functions.sink.OutputFormatSinkFunction.invoke(OutputFormatSinkFunction.java:87) org.apache.flink.streaming.api.functions.sink.SinkFunction.invoke(SinkFunction.java:49) org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:72) org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:112) org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:93) org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39) org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50) org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28) BatchCalc$597.processElement(Unknown Source) org.apache.flink.streaming.runtime.tasks.ChainingOutput.pushToOperator(ChainingOutput.java:112) org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:93) org.apache.flink.streaming.runtime.tasks.ChainingOutput.collect(ChainingOutput.java:39) org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:160) org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:110) org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:101) org.apache.flink.connector.file.src.impl.FileSourceRecordEmitter.emitRecord(FileSourceRecordEmitter.java:45) org.apache.flink.connector.file.src.impl.FileSourceRecordEmitter.emitRecord(FileSourceRecordEmitter.java:35) org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:128) org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:275) org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:67) org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:395) org.apache.flink.streaming.runtime.tasks.StreamTask$$Lambda$267/2023964146.runDefaultAction(Unknown Source) org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:191) org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:609) org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:573) org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:755) org.apache.flink.runtime.taskmanager.Task.run(Task.java:570) java.lang.Thread.run(Thread.java:748) 2021-01-31 23:04:23,691 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Close JobManager connection for job 13bc5fa5addba5772e9425161b48a2e3. 2021-01-31 23:04:23,691 INFO org.apache.flink.runtime.taskmanager.Task [] - Attempting to fail task externally Source: HiveSource-snmpprobe.p_port_traffic_5m -> Calc(select=[binaryid AS id, ver, CAST(2021-01-31 21:45:00:TIMESTAMP(6)) AS coltime, CAST(in_octets) AS in_octets, CAST(out_octets) AS out_octets, CAST(bi_octets) AS bi_octets, CAST(unimax_octets) AS unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, CAST((() DATE_FORMAT _UTF-16LE'yyyy-MM-dd HH:mm:ss')) AS crtime], where=[(coltime = 2021-01-31 21:45:00:TIMESTAMP(9))]) -> Sink: Sink(table=[myhive.prod_mysql_zqzynetdb.p_port_traffic_5m], fields=[id, ver, coltime, in_octets, out_octets, bi_octets, unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, crtime]) (1/1)#0 (7f4ca2467b4b31e2476bb7f5b93f6d33). 2021-01-31 23:04:23,691 INFO org.apache.flink.runtime.taskmanager.Task [] - Task Source: HiveSource-snmpprobe.p_port_traffic_5m -> Calc(select=[binaryid AS id, ver, CAST(2021-01-31 21:45:00:TIMESTAMP(6)) AS coltime, CAST(in_octets) AS in_octets, CAST(out_octets) AS out_octets, CAST(bi_octets) AS bi_octets, CAST(unimax_octets) AS unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, CAST((() DATE_FORMAT _UTF-16LE'yyyy-MM-dd HH:mm:ss')) AS crtime], where=[(coltime = 2021-01-31 21:45:00:TIMESTAMP(9))]) -> Sink: Sink(table=[myhive.prod_mysql_zqzynetdb.p_port_traffic_5m], fields=[id, ver, coltime, in_octets, out_octets, bi_octets, unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, crtime]) (1/1)#0 is already in state CANCELING 2021-01-31 23:04:23,693 INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:22, state:ALLOCATED, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=172.000mb (180355070 bytes), taskOffHeapMemory=0 bytes, managedMemory=128.000mb (134217730 bytes), networkMemory=16.000mb (16777216 bytes)}, allocationId: b70921707d488586bce0319feb054ebc, jobId: 13bc5fa5addba5772e9425161b48a2e3). 2021-01-31 23:04:23,693 INFO org.apache.flink.runtime.taskmanager.Task [] - Attempting to fail task externally Source: HiveSource-snmpprobe.p_port_traffic_5m -> Calc(select=[binaryid AS id, ver, CAST(2021-01-31 21:45:00:TIMESTAMP(6)) AS coltime, CAST(in_octets) AS in_octets, CAST(out_octets) AS out_octets, CAST(bi_octets) AS bi_octets, CAST(unimax_octets) AS unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, CAST((() DATE_FORMAT _UTF-16LE'yyyy-MM-dd HH:mm:ss')) AS crtime], where=[(coltime = 2021-01-31 21:45:00:TIMESTAMP(9))]) -> Sink: Sink(table=[myhive.prod_mysql_zqzynetdb.p_port_traffic_5m], fields=[id, ver, coltime, in_octets, out_octets, bi_octets, unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, crtime]) (1/1)#0 (7f4ca2467b4b31e2476bb7f5b93f6d33). 2021-01-31 23:04:23,693 INFO org.apache.flink.runtime.taskmanager.Task [] - Task Source: HiveSource-snmpprobe.p_port_traffic_5m -> Calc(select=[binaryid AS id, ver, CAST(2021-01-31 21:45:00:TIMESTAMP(6)) AS coltime, CAST(in_octets) AS in_octets, CAST(out_octets) AS out_octets, CAST(bi_octets) AS bi_octets, CAST(unimax_octets) AS unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, CAST((() DATE_FORMAT _UTF-16LE'yyyy-MM-dd HH:mm:ss')) AS crtime], where=[(coltime = 2021-01-31 21:45:00:TIMESTAMP(9))]) -> Sink: Sink(table=[myhive.prod_mysql_zqzynetdb.p_port_traffic_5m], fields=[id, ver, coltime, in_octets, out_octets, bi_octets, unimax_octets, in_speed, out_speed, bi_speed, unimax_speed, in_util, out_util, bi_util, unimax_util, inout_ratio, bandwidth, origin, crtime]) (1/1)#0 is already in state CANCELING 2021-01-31 23:04:23,695 INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:22, state:RELEASING, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=172.000mb (180355070 bytes), taskOffHeapMemory=0 bytes, managedMemory=128.000mb (134217730 bytes), networkMemory=16.000mb (16777216 bytes)}, allocationId: b70921707d488586bce0319feb054ebc, jobId: 13bc5fa5addba5772e9425161b48a2e3). 2021-01-31 23:04:33,693 INFO org.apache.flink.runtime.blob.PermanentBlobCache [] - Shutting down BLOB cache 2021-01-31 23:04:33,693 INFO org.apache.flink.runtime.blob.TransientBlobCache [] - Shutting down BLOB cache 2021-01-31 23:04:33,693 INFO org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Shutting down TaskExecutorLocalStateStoresManager. 2021-01-31 23:04:33,695 INFO org.apache.flink.runtime.filecache.FileCache [] - removed file cache directory /tmp/flink-dist-cache-b62b4fe6-a247-48c2-b1c2-f813ac4d2a78 2021-01-31 23:04:33,697 INFO org.apache.flink.runtime.io.disk.FileChannelManagerImpl [] - FileChannelManager removed spill file directory /tmp/flink-io-a8733545-1457-42f4-892a-779141bc4ce5 2021-01-31 23:04:33,697 INFO org.apache.flink.runtime.io.disk.FileChannelManagerImpl [] - FileChannelManager removed spill file directory /tmp/flink-netty-shuffle-3889e83c-89e2-436e-8e30-2997eaf8cd21 -- Sent from: http://apache-flink.147419.n8.nabble.com/ |
Free forum by Nabble | Edit this page |