flink-sql 客户端采用execution.target: yarn-per-job 模式,如何指定提交的队列??

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

flink-sql 客户端采用execution.target: yarn-per-job 模式,如何指定提交的队列??

飞翔
1、因为采用yarn-session模式,内存溢出
2、改为yarn-perjob模式,报如下错误:  --分析 可能是队列没有指定,队列中没有足够的资源,申请资源超时
Caused by: java.util.concurrent.TimeoutException: Heartbeat of TaskManager with id container_e66_1616483562588_2358_01_000002(XXXX) timed out.
        at org.apache.flink.runtime.jobmaster.JobMaster$TaskManagerHeartbeatListener.notifyHeartbeatTimeout(JobMaster.java:1378) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
        at org.apache.flink.runtime.heartbeat.HeartbeatMonitorImpl.run(HeartbeatMonitorImpl.java:111) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_151]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_151]
        at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:442) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
        at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:209) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
        at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) ~[flink-dist_2.11-1.12.1.jar:1.12.1]
        at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:159) ~[flink-dist_2.11-1.12.1.jar:1.12.1]

3、flink-session 提前启动 nohup bin/yarn-session.sh  -jm 1024 -tm 1024 -s 2 -qu root.CBT -d &  可以指定队列
4、那么我采用yarn-perjob模式时,应该每次提交(FLINK-CDC)sql,自动创建flink-per -job集群,怎么指定队列,请各路大神们赐教
Reply | Threaded
Open this post in threaded view
|

Re: flink-sql 客户端采用execution.target: yarn-per-job 模式,如何指定提交的队列??

JasonLee
hi

可以通过在 flink-conf.yaml 配置文件中添加 yarn.application.queue 参数来设置




-----
Best Wishes
JasonLee
--
Sent from: http://apache-flink.147419.n8.nabble.com/
Best Wishes
JasonLee
Reply | Threaded
Open this post in threaded view
|

Re: flink-sql 客户端采用execution.target: yarn-per-job 模式,如何指定提交的队列??

HunterXHunter
In reply to this post by 飞翔
flink run -yD yarn.application.queue=xxxxx



--
Sent from: http://apache-flink.147419.n8.nabble.com/