Apache Flink 中文用户邮件列表

Application Mode部署作业到yarn,找不到.properties文件

Classic

List

Threaded

2 messages Options

Application Mode部署作业到yarn,找不到.properties文件

flink
processFunction程序，main()中采用ParameterTool读取resources文件夹下的pro.properties配置文件（kafka地址等）；IDEA本地执行完全OK，maven打成jar包后，采用yarn
application的方式部署作业， bin/flink run-application -t yarn-application
/opt/quickstart-0.1.jar ；作业失败，查看yarn的container日志发现如下错误：

Caused by: org.apache.flink.client.program.ProgramInvocationException: The
main method caused an error: Properties file
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/file:/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/quickstart-0.1.jar!/pro.properties
does not exist

我打开quickstart-0.1.jar压缩包，发现是有pro.properties文件的；采用的是官网quickstart项目down下来pom.xml模板

--
Sent from: http://apache-flink.147419.n8.nabble.com/

Re: Application Mode部署作业到yarn,找不到.properties文件

jobManager的完整报错日志如下：
2021-01-21 07:53:23,023 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------
2021-01-21 07:53:23,027 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Starting
YarnApplicationClusterEntryPoint (Version: 1.11.2, Scala: 2.11, Rev:fe36135,
Date:2020-09-09T16:19:03+02:00)
2021-01-21 07:53:23,027 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS
current user: root
2021-01-21 07:53:23,374 WARN org.apache.hadoop.util.NativeCodeLoader
[] - Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
2021-01-21 07:53:23,484 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Current
Hadoop/Kerberos user: root
2021-01-21 07:53:23,484 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM: Java
HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.231-b11
2021-01-21 07:53:23,484 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Maximum
heap size: 981 MiBytes
2021-01-21 07:53:23,484 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
JAVA_HOME: /usr/java/jdk1.8.0_231-amd64
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Hadoop
version: 2.8.3
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM
Options:
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xmx1073741824
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xms1073741824
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:MaxMetaspaceSize=268435456
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog.file=/usr/local/hadoop/logs/userlogs/application_1611044725922_0013/container_1611044725922_0013_01_000001/jobmanager.log
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j.configuration=file:log4j.properties
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j.configurationFile=file:log4j.properties
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Program
Arguments: (none)
2021-01-21 07:53:23,485 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Classpath:
:lib/flink-csv-1.11.2.jar:lib/flink-json-1.11.2.jar:lib/flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:lib/flink-shaded-zookeeper-3.4.14.jar:lib/flink-table-blink_2.11-1.11.2.jar:lib/flink-table_2.11-1.11.2.jar:lib/log4j-1.2-api-2.12.1.jar:lib/log4j-api-2.12.1.jar:lib/log4j-core-2.12.1.jar:lib/log4j-slf4j-impl-2.12.1.jar:quickstart-0.1.jar:flink-dist_2.11-1.11.2.jar:flink-conf.yaml::/usr/local/hadoop/etc/hadoop:/usr/local/hadoop/share/hadoop/common/hadoop-common-3.2.1-tests.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/hadoop-kms-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/hadoop-nfs-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/accessors-smart-1.2.jar:/usr/local/hadoop/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/usr/local/hadoop/share/hadoop/common/lib/asm-5.0.4.jar:/usr/local/hadoop/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/usr/local/hadoop/share/hadoop/common/lib/avro-1.7.7.jar:/usr/local/hadoop/share/hadoop/common/lib/checker-qual-2.5.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-codec-1.11.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-compress-1.18.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-io-2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-lang3-3.7.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-net-3.6.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-text-1.4.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-client-2.13.0.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-framework-2.13.0.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/usr/local/hadoop/share/hadoop/common/lib/dnsjava-2.1.7.jar:/usr/local/hadoop/share/hadoop/common/lib/error_prone_annotations-2.2.0.jar:/usr/local/hadoop/share/hadoop/common/lib/failureaccess-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-annotations-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-auth-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/usr/local/hadoop/share/hadoop/common/lib/httpclient-4.5.6.jar:/usr/local/hadoop/share/hadoop/common/lib/httpcore-4.4.10.jar:/usr/local/hadoop/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-annotations-2.9.8.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-core-2.9.8.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-databind-2.9.8.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-core-1.19.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-json-1.19.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-server-1.19.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-servlet-1.19.jar:/usr/local/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/common/lib/jsch-0.1.54.jar:/usr/local/hadoop/share/hadoop/common/lib/json-smart-2.3.jar:/usr/local/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jsr305-3.0.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-client-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-common-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-core-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-server-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerb-util-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerby-config-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerby-util-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/usr/local/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/common/lib/metrics-core-3.2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/netty-3.10.5.Final.jar:/usr/local/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/usr/local/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/common/lib/re2j-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/usr/local/hadoop/share/hadoop/common/lib/snappy-java-1.0.5.jar:/usr/local/hadoop/share/hadoop/common/lib/stax2-api-3.1.4.jar:/usr/local/hadoop/share/hadoop/common/lib/token-provider-1.0.1.jar:/usr/local/hadoop/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/usr/local/hadoop/share/hadoop/common/lib/zookeeper-3.4.13.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.2.1-tests.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.2.1-tests.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.1-tests.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-3.2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.1-tests.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/asm-5.0.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/avro-1.7.7.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-io-2.5.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-lang3-3.7.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-net-3.6.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-text-1.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/error_prone_annotations-2.2.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/gson-2.2.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/hadoop-annotations-3.2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/hadoop-auth-3.2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/httpclient-4.5.6.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/httpcore-4.4.10.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-annotations-2.9.8.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-core-2.9.8.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-databind-2.9.8.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jettison-1.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/json-smart-2.3.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/okio-1.6.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/re2j-1.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-submarine-3.2.1.jar:/usr/local/hadoop/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/usr/local/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop/share/hadoop/yarn/lib/bcpkix-jdk15on-1.60.jar:/usr/local/hadoop/share/hadoop/yarn/lib/bcprov-jdk15on-1.60.jar:/usr/local/hadoop/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/usr/local/hadoop/share/hadoop/yarn/lib/fst-2.50.jar:/usr/local/hadoop/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/usr/local/hadoop/share/hadoop/yarn/lib/guice-4.0.jar:/usr/local/hadoop/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/usr/local/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-base-2.9.8.jar:/usr/local/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.9.8.jar:/usr/local/hadoop/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.9.8.jar:/usr/local/hadoop/share/hadoop/yarn/lib/java-util-1.9.0.jar:/usr/local/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop/share/hadoop/yarn/lib/jersey-client-1.19.jar:/usr/local/hadoop/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/usr/local/hadoop/share/hadoop/yarn/lib/json-io-2.5.1.jar:/usr/local/hadoop/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/usr/local/hadoop/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/usr/local/hadoop/share/hadoop/yarn/lib/objenesis-1.0.jar:/usr/local/hadoop/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/usr/local/hadoop/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar
2021-01-21 07:53:23,487 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------
2021-01-21 07:53:23,489 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Registered
UNIX signal handlers for [TERM, HUP, INT]
2021-01-21 07:53:23,499 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - YARN
daemon is running as: root Yarn client user obtainer: root
2021-01-21 07:53:23,509 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.memory.process.size, 1728m
2021-01-21 07:53:23,509 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.execution.failover-strategy, region
2021-01-21 07:53:23,509 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: high-availability.cluster-id,
application_1611044725922_0013
2021-01-21 07:53:23,509 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.address, localhost
2021-01-21 07:53:23,509 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.target, yarn-application
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.memory.process.size, 1600m
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.port, 6123
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.savepoint.ignore-unclaimed-state, false
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.attached, true
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: internal.cluster.execution-mode, NORMAL
2021-01-21 07:53:23,510 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error
while trying to split key and value in configuration file
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/flink-conf.yaml:11:
"$internal.application.program-args: "
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.shutdown-on-attached-exit, false
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: pipeline.jars, file:/opt/quickstart-0.1.jar
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: parallelism.default, 1
2021-01-21 07:53:23,510 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.numberOfTaskSlots, 1
2021-01-21 07:53:23,510 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error
while trying to split key and value in configuration file
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/flink-conf.yaml:16:
"pipeline.classpaths: "
2021-01-21 07:53:23,511 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: $internal.deployment.config-dir,
/root/flink/flink/conf
2021-01-21 07:53:23,511 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: $internal.yarn.log-config-file,
/root/flink/flink/conf/log4j.properties
2021-01-21 07:53:23,533 WARN org.apache.flink.configuration.Configuration
[] - Config uses deprecated configuration key 'web.port' instead of proper
key 'rest.bind-port'
2021-01-21 07:53:23,546 INFO
org.apache.flink.runtime.clusterframework.BootstrapTools [] - Setting
directories for temporary files to:
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013
2021-01-21 07:53:23,881 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Starting
YarnApplicationClusterEntryPoint.
2021-01-21 07:53:23,909 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Install
default filesystem.
2021-01-21 07:53:23,974 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Install
security context.
2021-01-21 07:53:24,053 INFO
org.apache.flink.runtime.security.modules.HadoopModule [] - Hadoop
user set to root (auth:SIMPLE)
2021-01-21 07:53:24,084 INFO
org.apache.flink.runtime.security.modules.JaasModule [] - Jaas file
will be created as
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/jaas-9102827030872345223.conf.
2021-01-21 07:53:24,102 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Initializing cluster services.
2021-01-21 07:53:24,122 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Trying to
start actor system, external address slave01:0, bind address 0.0.0.0:0.
2021-01-21 07:53:24,830 INFO akka.event.slf4j.Slf4jLogger
[] - Slf4jLogger started
2021-01-21 07:53:24,891 INFO akka.remote.Remoting
[] - Starting remoting
2021-01-21 07:53:25,046 INFO akka.remote.Remoting
[] - Remoting started; listening on addresses
:[akka.tcp://flink@slave01:39703]
2021-01-21 07:53:25,812 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Actor
system started at akka.tcp://flink@slave01:39703
2021-01-21 07:53:25,848 WARN org.apache.flink.configuration.Configuration
[] - Config uses deprecated configuration key 'web.port' instead of proper
key 'rest.port'
2021-01-21 07:53:25,865 INFO org.apache.flink.runtime.blob.BlobServer
[] - Created BLOB server storage directory
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/blobStore-50ff2762-e8e7-4272-a741-7f358712d9d2
2021-01-21 07:53:25,872 INFO org.apache.flink.runtime.blob.BlobServer
[] - Started BLOB server at 0.0.0.0:41303 - max concurrent requests: 50 -
max backlog: 1000
2021-01-21 07:53:25,896 INFO
org.apache.flink.runtime.metrics.MetricRegistryImpl [] - No metrics
reporter configured, no metrics will be exposed/reported.
2021-01-21 07:53:25,900 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Trying to
start actor system, external address slave01:0, bind address 0.0.0.0:0.
2021-01-21 07:53:25,934 INFO akka.event.slf4j.Slf4jLogger
[] - Slf4jLogger started
2021-01-21 07:53:25,940 INFO akka.remote.Remoting
[] - Starting remoting
2021-01-21 07:53:25,949 INFO akka.remote.Remoting
[] - Remoting started; listening on addresses
:[akka.tcp://flink-metrics@slave01:34458]
2021-01-21 07:53:26,038 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Actor
system started at akka.tcp://flink-metrics@slave01:34458
2021-01-21 07:53:26,184 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Starting
RPC endpoint for org.apache.flink.runtime.metrics.dump.MetricQueryService at
akka://flink-metrics/user/rpc/MetricQueryService .
2021-01-21 07:53:26,245 WARN org.apache.flink.configuration.Configuration
[] - Config uses deprecated configuration key 'web.port' instead of proper
key 'rest.bind-port'
2021-01-21 07:53:26,246 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Upload
directory
/tmp/flink-web-4753591c-1dba-44da-aff6-8c4af6c99eb7/flink-web-upload does
not exist.
2021-01-21 07:53:26,249 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Created
directory
/tmp/flink-web-4753591c-1dba-44da-aff6-8c4af6c99eb7/flink-web-upload for
file uploads.
2021-01-21 07:53:26,276 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Starting
rest endpoint.
2021-01-21 07:53:26,926 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils [] - Determined
location of main cluster component log file:
/usr/local/hadoop/logs/userlogs/application_1611044725922_0013/container_1611044725922_0013_01_000001/jobmanager.log
2021-01-21 07:53:26,926 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils [] - Determined
location of main cluster component stdout file:
/usr/local/hadoop/logs/userlogs/application_1611044725922_0013/container_1611044725922_0013_01_000001/jobmanager.out
2021-01-21 07:53:27,535 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Rest
endpoint listening at slave01:33441
2021-01-21 07:53:27,544 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
http://slave01:33441 was granted leadership with
leaderSessionID=00000000-0000-0000-0000-000000000000
2021-01-21 07:53:27,546 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Web
frontend listening at http://slave01:33441.
2021-01-21 07:53:27,875 INFO
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The
derived from fraction jvm overhead memory (172.800mb (181193935 bytes)) is
less than its min value 192.000mb (201326592 bytes), min value will be used
instead
2021-01-21 07:53:27,958 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Starting
RPC endpoint for org.apache.flink.yarn.YarnResourceManager at
akka://flink/user/rpc/resourcemanager_0 .
2021-01-21 07:53:27,991 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.memory.process.size, 1728m
2021-01-21 07:53:27,991 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.execution.failover-strategy, region
2021-01-21 07:53:27,991 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: high-availability.cluster-id,
application_1611044725922_0013
2021-01-21 07:53:27,991 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.address, localhost
2021-01-21 07:53:27,991 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.target, yarn-application
2021-01-21 07:53:27,991 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.memory.process.size, 1600m
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.port, 6123
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.savepoint.ignore-unclaimed-state, false
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.attached, true
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: internal.cluster.execution-mode, NORMAL
2021-01-21 07:53:27,992 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error
while trying to split key and value in configuration file
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/flink-conf.yaml:11:
"$internal.application.program-args: "
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.shutdown-on-attached-exit, false
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: pipeline.jars, file:/opt/quickstart-0.1.jar
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: parallelism.default, 1
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.numberOfTaskSlots, 1
2021-01-21 07:53:27,992 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error
while trying to split key and value in configuration file
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/flink-conf.yaml:16:
"pipeline.classpaths: "
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: $internal.deployment.config-dir,
/root/flink/flink/conf
2021-01-21 07:53:27,992 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: $internal.yarn.log-config-file,
/root/flink/flink/conf/log4j.properties
2021-01-21 07:53:28,017 INFO
org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled
external resources: []
2021-01-21 07:53:28,036 INFO
org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess []
- Start SessionDispatcherLeaderProcess.
2021-01-21 07:53:28,041 INFO
org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess []
- Recover all persisted job graphs.
2021-01-21 07:53:28,042 INFO
org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess []
- Successfully recovered 0 persisted job graphs.
2021-01-21 07:53:28,079 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Starting
RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at
akka://flink/user/rpc/dispatcher_1 .
2021-01-21 07:53:28,152 INFO org.apache.flink.client.ClientUtils
[] - Starting program (detached: false)
2021-01-21 07:53:28,171 INFO org.apache.hadoop.yarn.client.RMProxy
[] - Connecting to ResourceManager at master/172.10.0.10:8030
2021-01-21 07:53:28,215 WARN
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap
[] - Application FAILED:
java.util.concurrent.CompletionException:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
~[?:1.8.0_231]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:245)
~[quickstart-0.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:199)
~[quickstart-0.1.jar:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_231]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_231]
at
org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:154)
[quickstart-0.1.jar:?]
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
[quickstart-0.1.jar:?]
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
[quickstart-0.1.jar:?]
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
[quickstart-0.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
[quickstart-0.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
[quickstart-0.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[quickstart-0.1.jar:?]
Caused by:
org.apache.flink.client.deployment.application.ApplicationExecutionException:
Could not execute application.
... 11 more
Caused by: org.apache.flink.client.program.ProgramInvocationException: The
main method caused an error: Properties file
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/file:/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/quickstart-0.1.jar!/pro.properties
does not exist
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:302)
~[quickstart-0.1.jar:?]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)
~[quickstart-0.1.jar:?]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149)
~[quickstart-0.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:230)
~[quickstart-0.1.jar:?]
... 10 more
Caused by: java.io.FileNotFoundException: Properties file
/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/file:/root/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1611044725922_0013/container_1611044725922_0013_01_000001/quickstart-0.1.jar!/pro.properties
does not exist
at
org.apache.flink.api.java.utils.ParameterTool.fromPropertiesFile(ParameterTool.java:120)
~[quickstart-0.1.jar:?]
at
org.apache.flink.api.java.utils.ParameterTool.fromPropertiesFile(ParameterTool.java:107)
~[quickstart-0.1.jar:?]
at org.myorg.quickstart.Sink_KafkaSink.main(Sink_KafkaSink.java:26)
~[quickstart-0.1.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_231]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_231]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_231]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231]
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)
~[quickstart-0.1.jar:?]
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)
~[quickstart-0.1.jar:?]
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149)
~[quickstart-0.1.jar:?]
at
org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:230)
~[quickstart-0.1.jar:?]
... 10 more
2021-01-21 07:53:28,235 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Shutting
YarnApplicationClusterEntryPoint down with application status FAILED.
Diagnostics null.
2021-01-21 07:53:28,235 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Shutting
down rest endpoint.
2021-01-21 07:53:28,281 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Removing
cache directory
/tmp/flink-web-4753591c-1dba-44da-aff6-8c4af6c99eb7/flink-web-ui
2021-01-21 07:53:28,288 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
http://slave01:33441 lost leadership
2021-01-21 07:53:28,288 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Shut down
complete.
2021-01-21 07:53:28,420 INFO org.apache.flink.yarn.YarnResourceManager
[] - Recovered 0 containers from previous attempts ([]).
2021-01-21 07:53:28,421 INFO org.apache.flink.yarn.YarnResourceManager
[] - Register application master response contains scheduler resource types:
[MEMORY].
2021-01-21 07:53:28,421 INFO org.apache.flink.yarn.YarnResourceManager
[] - Container matching strategy: IGNORE_VCORE.
2021-01-21 07:53:28,426 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Upper
bound of the thread pool size is 500
2021-01-21 07:53:28,427 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] -
yarn.client.max-cached-nodemanagers-proxies : 0
2021-01-21 07:53:28,444 INFO
org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent
[] - Closing components.
2021-01-21 07:53:28,462 INFO
org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess []
- Stopping SessionDispatcherLeaderProcess.
2021-01-21 07:53:28,463 INFO org.apache.flink.yarn.YarnResourceManager
[] - ResourceManager
akka.tcp://flink@slave01:39703/user/rpc/resourcemanager_0 was granted
leadership with fencing token 00000000000000000000000000000000
2021-01-21 07:53:28,465 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopping
dispatcher akka.tcp://flink@slave01:39703/user/rpc/dispatcher_1.
2021-01-21 07:53:28,465 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopping
all currently running jobs of dispatcher
akka.tcp://flink@slave01:39703/user/rpc/dispatcher_1.
2021-01-21 07:53:28,465 INFO
org.apache.flink.runtime.rest.handler.legacy.backpressure.BackPressureRequestCoordinator
[] - Shutting down back pressure request coordinator.
2021-01-21 07:53:28,466 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopped
dispatcher akka.tcp://flink@slave01:39703/user/rpc/dispatcher_1.
2021-01-21 07:53:28,491 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] -
Starting the SlotManager.
2021-01-21 07:53:28,508 WARN org.apache.hadoop.ipc.Client
[] - interrupted waiting to send rpc request to server
java.lang.InterruptedException: null
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
~[?:1.8.0_231]
at java.util.concurrent.FutureTask.get(FutureTask.java:191)
~[?:1.8.0_231]
at
org.apache.hadoop.ipc.Client$Connection.sendRpcRequest(Client.java:1138)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at org.apache.hadoop.ipc.Client.call(Client.java:1387)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at org.apache.hadoop.ipc.Client.call(Client.java:1345)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at com.sun.proxy.$Proxy32.allocate(Unknown Source) [?:?]
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.allocate(ApplicationMasterProtocolPBClientImpl.java:77)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_231]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_231]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_231]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at com.sun.proxy.$Proxy33.allocate(Unknown Source) [?:?]
at
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.allocate(AMRMClientImpl.java:289)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:273)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
2021-01-21 07:53:28,513 ERROR
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl [] -
Exception on heartbeat
java.io.IOException: java.lang.InterruptedException
at org.apache.hadoop.ipc.Client.call(Client.java:1393)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at org.apache.hadoop.ipc.Client.call(Client.java:1345)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at com.sun.proxy.$Proxy32.allocate(Unknown Source) ~[?:?]
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.allocate(ApplicationMasterProtocolPBClientImpl.java:77)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_231]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_231]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_231]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:409)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:346)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at com.sun.proxy.$Proxy33.allocate(Unknown Source) ~[?:?]
at
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.allocate(AMRMClientImpl.java:289)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$HeartbeatThread.run(AMRMClientAsyncImpl.java:273)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
Caused by: java.lang.InterruptedException
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
~[?:1.8.0_231]
at java.util.concurrent.FutureTask.get(FutureTask.java:191)
~[?:1.8.0_231]
at
org.apache.hadoop.ipc.Client$Connection.sendRpcRequest(Client.java:1138)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
at org.apache.hadoop.ipc.Client.call(Client.java:1387)
~[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
... 17 more
2021-01-21 07:53:28,513 INFO
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl [] -
Interrupted while waiting for queue
java.lang.InterruptedException: null
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
~[?:1.8.0_231]
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
~[?:1.8.0_231]
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
~[?:1.8.0_231]
at
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:323)
[flink-shaded-hadoop-2-uber-2.8.3-8.0.jar:2.8.3-8.0]
2021-01-21 07:53:28,615 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] -
Closing the SlotManager.
2021-01-21 07:53:28,615 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] -
Suspending the SlotManager.
2021-01-21 07:53:28,628 INFO org.apache.flink.runtime.blob.BlobServer
[] - Stopped BLOB server at 0.0.0.0:41303
2021-01-21 07:53:28,631 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Stopping
Akka RPC service.
2021-01-21 07:53:28,639 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Stopping
Akka RPC service.
2021-01-21 07:53:28,687 INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Shutting
down remote daemon.
2021-01-21 07:53:28,742 INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Shutting
down remote daemon.
2021-01-21 07:53:28,742 INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remote
daemon shut down; proceeding with flushing remote transports.
2021-01-21 07:53:28,745 INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remote
daemon shut down; proceeding with flushing remote transports.
2021-01-21 07:53:28,780 INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remoting
shut down.
2021-01-21 07:53:28,807 INFO
akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remoting
shut down.
2021-01-21 07:53:28,817 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Stopped
Akka RPC service.
2021-01-21 07:53:28,822 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
Terminating cluster entrypoint process YarnApplicationClusterEntryPoint with
exit code 2.
java.util.concurrent.CompletionException:
org.apache.flink.runtime.rpc.exceptions.FencingTokenException: Fencing token
not set: Ignoring message LocalFencedMessage(null,
LocalRpcInvocation(deregisterApplication(ApplicationStatus, String))) sent
to akka.tcp://flink@slave01:39703/user/rpc/resourcemanager_0 because the
fencing token is null.
at
java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
~[?:1.8.0_231]
at
org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:227)
~[quickstart-0.1.jar:?]
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
~[?:1.8.0_231]
at
java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
~[?:1.8.0_231]
at
org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:890)
~[quickstart-0.1.jar:?]
at akka.dispatch.OnComplete.internal(Future.scala:263)
~[quickstart-0.1.jar:?]
at akka.dispatch.OnComplete.internal(Future.scala:261)
~[quickstart-0.1.jar:?]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
~[quickstart-0.1.jar:?]
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
~[quickstart-0.1.jar:?]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
~[quickstart-0.1.jar:?]
at
org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:74)
~[quickstart-0.1.jar:?]
at
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
~[quickstart-0.1.jar:?]
at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
~[quickstart-0.1.jar:?]
at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
~[quickstart-0.1.jar:?]
at akka.actor.ActorRef.tell(ActorRef.scala:126)
~[quickstart-0.1.jar:?]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.sendErrorIfSender(AkkaRpcActor.java:440)
~[quickstart-0.1.jar:?]
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:61)
~[quickstart-0.1.jar:?]
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
~[quickstart-0.1.jar:?]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
~[quickstart-0.1.jar:?]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
~[quickstart-0.1.jar:?]
at
scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
~[quickstart-0.1.jar:?]
at
akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
~[quickstart-0.1.jar:?]
at
scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
~[quickstart-0.1.jar:?]
at
scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
~[quickstart-0.1.jar:?]
at
scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
~[quickstart-0.1.jar:?]
at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
~[quickstart-0.1.jar:?]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
~[quickstart-0.1.jar:?]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
~[quickstart-0.1.jar:?]
at akka.actor.ActorCell.invoke(ActorCell.scala:561)
~[quickstart-0.1.jar:?]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
~[quickstart-0.1.jar:?]
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
~[quickstart-0.1.jar:?]
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
~[quickstart-0.1.jar:?]
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
~[quickstart-0.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
~[quickstart-0.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
~[quickstart-0.1.jar:?]
at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
~[quickstart-0.1.jar:?]
Caused by: org.apache.flink.runtime.rpc.exceptions.FencingTokenException:
Fencing token not set: Ignoring message LocalFencedMessage(null,
LocalRpcInvocation(deregisterApplication(ApplicationStatus, String))) sent
to akka.tcp://flink@slave01:39703/user/rpc/resourcemanager_0 because the
fencing token is null.
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:63)
~[quickstart-0.1.jar:?]
... 19 more

--
Sent from: http://apache-flink.147419.n8.nabble.com/