flink1.11 sink hive error

[Zhou Zach]

hi all,
flink1.11 sql sink hive table 报错：


java.util.concurrent.CompletionException: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application.
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_161]
        at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_161]
        at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943) ~[?:1.8.0_161]
        at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_161]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_161]
        at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_161]
        at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:245) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:199) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_161]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_161]
        at org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:154) [data-flow-1.0.jar:?]
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) [qile-data-flow-1.0.jar:?]
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44) [data-flow-1.0.jar:?]
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [qile-data-flow-1.0.jar:?]
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [data-flow-1.0.jar:?]
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [data-flow-1.0.jar:?]
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [data-flow-1.0.jar:?]
Caused by: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application.
        ... 11 more
Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Embedded metastore is not allowed. Make sure you have set a valid value for hive.metastore.uris
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:302) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:230) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        ... 10 more
Caused by: java.lang.IllegalArgumentException: Embedded metastore is not allowed. Make sure you have set a valid value for hive.metastore.uris
        at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:139) ~[data-flow-1.0.jar:?]
        at org.apache.flink.table.catalog.hive.HiveCatalog.<init>(HiveCatalog.java:171) ~[flink-sql-connector-hive-2.2.0_2.11-1.11.0.jar:1.11.0]
        at org.apache.flink.table.catalog.hive.HiveCatalog.<init>(HiveCatalog.java:157) ~[flink-sql-connector-hive-2.2.0_2.11-1.11.0.jar:1.11.0]
        at cn.ibobei.qile.dataflow.sql.FromKafkaSinkHiveAndHbase$.main(FromKafkaSinkHiveAndHbase.scala:27) ~[data-flow-1.0.jar:?]
        at cn.ibobei.qile.dataflow.sql.FromKafkaSinkHiveAndHbase.main(FromKafkaSinkHiveAndHbase.scala) ~[data-flow-1.0.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_161]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_161]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_161]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_161]
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:230) ~[flink-clients_2.11-1.11.0.jar:1.11.0]
        ... 10 more

[Leonard Xu]

Hello


> 在 2020年7月14日，19:52，Zhou Zach <[hidden email]> 写道：
>
> : Embedded metastore is not allowed.

Flink 集成 Hive 时，不支持 embedded metastore 的, 你需要起一个hive metastore 并在conf文件配置 hive.metastore.uris, 支持的 metastore 版本 参考[1]

Best,
Leonard Xu
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar <https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar>

[Zhou Zach]

Hi，


是在flink的conf文件配置hive.metastore.uris吗

















在 2020-07-14 20:03:11，"Leonard Xu" <[hidden email]> 写道：

>Hello
>
>
>> 在 2020年7月14日，19:52，Zhou Zach <[hidden email]> 写道：
>>
>> : Embedded metastore is not allowed.
>
>Flink 集成 Hive 时，不支持 embedded metastore 的, 你需要起一个hive metastore 并在conf文件配置 hive.metastore.uris, 支持的 metastore 版本 参考[1]
>
>Best,
>Leonard Xu
>[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar <https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar>

[Leonard Xu]

Hi,
你安装 hive 的 metastore 后，在你 hivehome/conf/hive-site.xml 文件中添加这样一个配置：
  <property>
    <name>hive.metastore.uris</name>
    <value>thrift://xxxx:9083</value>
    <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
  </property>
一般生产环境应该也是这样配置，
然后 Flink 对接到hive配置参考[1]，应该和你之前用的没啥变化，就是不支持 embedded 的 metastore

祝好，
Leonard Xu
[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#connecting-to-hive <https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#connecting-to-hive>

> 在 2020年7月14日，20:29，Zhou Zach <[hidden email]> 写道：
>
> Hi，
>
>
> 是在flink的conf文件配置hive.metastore.uris吗
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> 在 2020-07-14 20:03:11，"Leonard Xu" <[hidden email]> 写道：
>> Hello
>>
>>
>>> 在 2020年7月14日，19:52，Zhou Zach <[hidden email]> 写道：
>>>
>>> : Embedded metastore is not allowed.
>>
>> Flink 集成 Hive 时，不支持 embedded metastore 的, 你需要起一个hive metastore 并在conf文件配置 hive.metastore.uris, 支持的 metastore 版本 参考[1]
>>
>> Best,
>> Leonard Xu
>> [1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar <https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar>

[Zhou Zach]

Hi,


我刚才把flink sink的hive table，hive hdfs目录都删了，hbase表数据也清空了（hbase 通过hue hive table方式查询），然后重启程序，就可以了，
等再出问题，我试下你这种方法，感谢答疑！

















在 2020-07-14 20:42:16，"Leonard Xu" <[hidden email]> 写道：

>Hi,
>你安装 hive 的 metastore 后，在你 hivehome/conf/hive-site.xml 文件中添加这样一个配置：
>  <property>
>    <name>hive.metastore.uris</name>
>    <value>thrift://xxxx:9083</value>
>    <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
>  </property>
>一般生产环境应该也是这样配置，
>然后 Flink 对接到hive配置参考[1]，应该和你之前用的没啥变化，就是不支持 embedded 的 metastore
>
>祝好，
>Leonard Xu
>[1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#connecting-to-hive <https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#connecting-to-hive>
>
>> 在 2020年7月14日，20:29，Zhou Zach <[hidden email]> 写道：
>>
>> Hi，
>>
>>
>> 是在flink的conf文件配置hive.metastore.uris吗
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> 在 2020-07-14 20:03:11，"Leonard Xu" <[hidden email]> 写道：
>>> Hello
>>>
>>>
>>>> 在 2020年7月14日，19:52，Zhou Zach <[hidden email]> 写道：
>>>>
>>>> : Embedded metastore is not allowed.
>>>
>>> Flink 集成 Hive 时，不支持 embedded metastore 的, 你需要起一个hive metastore 并在conf文件配置 hive.metastore.uris, 支持的 metastore 版本 参考[1]
>>>
>>> Best,
>>> Leonard Xu
>>> [1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar <https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive/#using-bundled-hive-jar>
>