按照官网进行flink-shell操作，出现无法解决的错误：Only BatchTableSource and InputFormatTableSource are supported in BatchTableEnvironment.

[张锴]

具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。

scala> import org.apache.flink.table.catalog.hive.HiveCatalog
import org.apache.flink.table.catalog.hive.HiveCatalog

scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
"/opt/hive2.3.3/conf", "2.3.3");
hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
org.apache.flink.table.catalog.hive.HiveCatalog@193f3306

scala> btenv.registerCatalog("hive", hiveCatalog)
Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
server's identity verification is not recommended. According to MySQL
5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established
by default if explicit option isn't set. For compliance with existing
applications not using SSL the verifyServerCertificate property is set to
'false'. You need either to explicitly disable SSL by setting useSSL=false,
or set useSSL=true and provide truststore for server certificate
verification.
Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
server's identity verification is not recommended. According to MySQL
5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established
by default if explicit option isn't set. For compliance with existing
applications not using SSL the verifyServerCertificate property is set to
'false'. You need either to explicitly disable SSL by setting useSSL=false,
or set useSSL=true and provide truststore for server certificate
verification.

scala> btenv.useCatalog("hive")

scala> btenv.listTables
res2: Array[String] = Array(mytable)

scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
org.apache.flink.table.api.TableException: Only BatchTableSource and
InputFormatTableSource are supported in BatchTableEnvironment.
  at
org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
  at
org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
  at
org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
  at
org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
  at
org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
  ... 30 elided

[Jingsong Li]

Hi,

不好意思，现在版本hive connector已经不支持old planner了，
但是scala shell还是默认old planner。

Best,
Jingsong Lee

On Thu, May 21, 2020 at 3:24 PM 张锴 <[hidden email]> wrote:

> 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
>
> scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> import org.apache.flink.table.catalog.hive.HiveCatalog
>
> scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> "/opt/hive2.3.3/conf", "2.3.3");
> hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
>
> scala> btenv.registerCatalog("hive", hiveCatalog)
> Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> server's identity verification is not recommended. According to MySQL
> 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established
> by default if explicit option isn't set. For compliance with existing
> applications not using SSL the verifyServerCertificate property is set to
> 'false'. You need either to explicitly disable SSL by setting useSSL=false,
> or set useSSL=true and provide truststore for server certificate
> verification.
> Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> server's identity verification is not recommended. According to MySQL
> 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established
> by default if explicit option isn't set. For compliance with existing
> applications not using SSL the verifyServerCertificate property is set to
> 'false'. You need either to explicitly disable SSL by setting useSSL=false,
> or set useSSL=true and provide truststore for server certificate
> verification.
>
> scala> btenv.useCatalog("hive")
>
> scala> btenv.listTables
> res2: Array[String] = Array(mytable)
>
> scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> org.apache.flink.table.api.TableException: Only BatchTableSource and
> InputFormatTableSource are supported in BatchTableEnvironment.
>   at
>
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
>   at
>
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
>   at
>
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
>   at
>
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
>   at
>
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
>   ... 30 elided
>

--
Best, Jingsong Lee

[Jeff Zhang]

可以在zeppelin里写scala代码，是支持hive的，参考这个视频，
https://www.bilibili.com/video/BV1Te411W73b?p=10

也可以到这个钉钉群讨论: 30022475

Jingsong Li <[hidden email]> 于2020年5月21日周四 下午4:43写道：

> Hi,
>
> 不好意思，现在版本hive connector已经不支持old planner了，
> 但是scala shell还是默认old planner。
>
> Best,
> Jingsong Lee
>
> On Thu, May 21, 2020 at 3:24 PM 张锴 <[hidden email]> wrote:
>
> > 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
> >
> > scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> > import org.apache.flink.table.catalog.hive.HiveCatalog
> >
> > scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> > "/opt/hive2.3.3/conf", "2.3.3");
> > hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> > org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
> >
> > scala> btenv.registerCatalog("hive", hiveCatalog)
> > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > server's identity verification is not recommended. According to MySQL
> > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> established
> > by default if explicit option isn't set. For compliance with existing
> > applications not using SSL the verifyServerCertificate property is set to
> > 'false'. You need either to explicitly disable SSL by setting
> useSSL=false,
> > or set useSSL=true and provide truststore for server certificate
> > verification.
> > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > server's identity verification is not recommended. According to MySQL
> > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> established
> > by default if explicit option isn't set. For compliance with existing
> > applications not using SSL the verifyServerCertificate property is set to
> > 'false'. You need either to explicitly disable SSL by setting
> useSSL=false,
> > or set useSSL=true and provide truststore for server certificate
> > verification.
> >
> > scala> btenv.useCatalog("hive")
> >
> > scala> btenv.listTables
> > res2: Array[String] = Array(mytable)
> >
> > scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> > org.apache.flink.table.api.TableException: Only BatchTableSource and
> > InputFormatTableSource are supported in BatchTableEnvironment.
> >   at
> >
> >
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> >   at
> >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> >   at
> >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> >   at
> >
> >
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> >   at
> >
> >
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> >   ... 30 elided
> >
>
>
> --
> Best, Jingsong Lee
>

--
Best Regards

Jeff Zhang

[张锴]

我想请问，在blink planner中，通过连接hive的方式将数据转成DataSet[Row]呢


Jingsong Li <[hidden email]> 于2020年5月21日周四 下午4:43写道：

> Hi,
>
> 不好意思，现在版本hive connector已经不支持old planner了，
> 但是scala shell还是默认old planner。
>
> Best,
> Jingsong Lee
>
> On Thu, May 21, 2020 at 3:24 PM 张锴 <[hidden email]> wrote:
>
> > 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
> >
> > scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> > import org.apache.flink.table.catalog.hive.HiveCatalog
> >
> > scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> > "/opt/hive2.3.3/conf", "2.3.3");
> > hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> > org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
> >
> > scala> btenv.registerCatalog("hive", hiveCatalog)
> > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > server's identity verification is not recommended. According to MySQL
> > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> established
> > by default if explicit option isn't set. For compliance with existing
> > applications not using SSL the verifyServerCertificate property is set to
> > 'false'. You need either to explicitly disable SSL by setting
> useSSL=false,
> > or set useSSL=true and provide truststore for server certificate
> > verification.
> > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > server's identity verification is not recommended. According to MySQL
> > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> established
> > by default if explicit option isn't set. For compliance with existing
> > applications not using SSL the verifyServerCertificate property is set to
> > 'false'. You need either to explicitly disable SSL by setting
> useSSL=false,
> > or set useSSL=true and provide truststore for server certificate
> > verification.
> >
> > scala> btenv.useCatalog("hive")
> >
> > scala> btenv.listTables
> > res2: Array[String] = Array(mytable)
> >
> > scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> > org.apache.flink.table.api.TableException: Only BatchTableSource and
> > InputFormatTableSource are supported in BatchTableEnvironment.
> >   at
> >
> >
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> >   at
> >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> >   at
> >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> >   at
> >
> >
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> >   at
> >
> >
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> >   ... 30 elided
> >
>
>
> --
> Best, Jingsong Lee
>

[张锴]

我去看看

Jeff Zhang <[hidden email]> 于2020年5月21日周四 下午4:54写道：

> 可以在zeppelin里写scala代码，是支持hive的，参考这个视频，
> https://www.bilibili.com/video/BV1Te411W73b?p=10
>
> 也可以到这个钉钉群讨论: 30022475
>
> Jingsong Li <[hidden email]> 于2020年5月21日周四 下午4:43写道：
>
> > Hi,
> >
> > 不好意思，现在版本hive connector已经不支持old planner了，
> > 但是scala shell还是默认old planner。
> >
> > Best,
> > Jingsong Lee
> >
> > On Thu, May 21, 2020 at 3:24 PM 张锴 <[hidden email]> wrote:
> >
> > > 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
> > >
> > > scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> > > import org.apache.flink.table.catalog.hive.HiveCatalog
> > >
> > > scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> > > "/opt/hive2.3.3/conf", "2.3.3");
> > > hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> > > org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
> > >
> > > scala> btenv.registerCatalog("hive", hiveCatalog)
> > > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > > server's identity verification is not recommended. According to MySQL
> > > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> > established
> > > by default if explicit option isn't set. For compliance with existing
> > > applications not using SSL the verifyServerCertificate property is set
> to
> > > 'false'. You need either to explicitly disable SSL by setting
> > useSSL=false,
> > > or set useSSL=true and provide truststore for server certificate
> > > verification.
> > > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection without
> > > server's identity verification is not recommended. According to MySQL
> > > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> > established
> > > by default if explicit option isn't set. For compliance with existing
> > > applications not using SSL the verifyServerCertificate property is set
> to
> > > 'false'. You need either to explicitly disable SSL by setting
> > useSSL=false,
> > > or set useSSL=true and provide truststore for server certificate
> > > verification.
> > >
> > > scala> btenv.useCatalog("hive")
> > >
> > > scala> btenv.listTables
> > > res2: Array[String] = Array(mytable)
> > >
> > > scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> > > org.apache.flink.table.api.TableException: Only BatchTableSource and
> > > InputFormatTableSource are supported in BatchTableEnvironment.
> > >   at
> > >
> > >
> >
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> > >   at
> > >
> > >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> > >   at
> > >
> > >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> > >   at
> > >
> > >
> >
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> > >   at
> > >
> > >
> >
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> > >   ... 30 elided
> > >
> >
> >
> > --
> > Best, Jingsong Lee
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

[Jingsong Li]

> 在blink planner中，通过连接hive的方式将数据转成DataSet[Row]呢
不能，社区正在做BoundedStream，在stream上完成DataSet的功能。

Best,
Jingsong Lee

On Thu, May 21, 2020 at 6:45 PM 张锴 <[hidden email]> wrote:

> 我去看看
>
> Jeff Zhang <[hidden email]> 于2020年5月21日周四 下午4:54写道：
>
> > 可以在zeppelin里写scala代码，是支持hive的，参考这个视频，
> > https://www.bilibili.com/video/BV1Te411W73b?p=10
> >
> > 也可以到这个钉钉群讨论: 30022475
> >
> > Jingsong Li <[hidden email]> 于2020年5月21日周四 下午4:43写道：
> >
> > > Hi,
> > >
> > > 不好意思，现在版本hive connector已经不支持old planner了，
> > > 但是scala shell还是默认old planner。
> > >
> > > Best,
> > > Jingsong Lee
> > >
> > > On Thu, May 21, 2020 at 3:24 PM 张锴 <[hidden email]> wrote:
> > >
> > > > 具体操作及错误信息我贴到下面，各位大佬帮忙看下如何解决，不知道是不是BUG。
> > > >
> > > > scala> import org.apache.flink.table.catalog.hive.HiveCatalog
> > > > import org.apache.flink.table.catalog.hive.HiveCatalog
> > > >
> > > > scala> val hiveCatalog = new HiveCatalog("hive", "mydatabase",
> > > > "/opt/hive2.3.3/conf", "2.3.3");
> > > > hiveCatalog: org.apache.flink.table.catalog.hive.HiveCatalog =
> > > > org.apache.flink.table.catalog.hive.HiveCatalog@193f3306
> > > >
> > > > scala> btenv.registerCatalog("hive", hiveCatalog)
> > > > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection
> without
> > > > server's identity verification is not recommended. According to MySQL
> > > > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> > > established
> > > > by default if explicit option isn't set. For compliance with existing
> > > > applications not using SSL the verifyServerCertificate property is
> set
> > to
> > > > 'false'. You need either to explicitly disable SSL by setting
> > > useSSL=false,
> > > > or set useSSL=true and provide truststore for server certificate
> > > > verification.
> > > > Thu May 21 15:10:05 CST 2020 WARN: Establishing SSL connection
> without
> > > > server's identity verification is not recommended. According to MySQL
> > > > 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be
> > > established
> > > > by default if explicit option isn't set. For compliance with existing
> > > > applications not using SSL the verifyServerCertificate property is
> set
> > to
> > > > 'false'. You need either to explicitly disable SSL by setting
> > > useSSL=false,
> > > > or set useSSL=true and provide truststore for server certificate
> > > > verification.
> > > >
> > > > scala> btenv.useCatalog("hive")
> > > >
> > > > scala> btenv.listTables
> > > > res2: Array[String] = Array(mytable)
> > > >
> > > > scala> btenv.sqlQuery("select * from mytable").toDataSet[Row].print()
> > > > org.apache.flink.table.api.TableException: Only BatchTableSource and
> > > > InputFormatTableSource are supported in BatchTableEnvironment.
> > > >   at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.plan.nodes.dataset.BatchTableSourceScan.translateToPlan(BatchTableSourceScan.scala:118)
> > > >   at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:306)
> > > >   at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.api.internal.BatchTableEnvImpl.translate(BatchTableEnvImpl.scala:281)
> > > >   at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.api.scala.internal.BatchTableEnvironmentImpl.toDataSet(BatchTableEnvironmentImpl.scala:69)
> > > >   at
> > > >
> > > >
> > >
> >
> org.apache.flink.table.api.scala.TableConversions.toDataSet(TableConversions.scala:53)
> > > >   ... 30 elided
> > > >
> > >
> > >
> > > --
> > > Best, Jingsong Lee
> > >
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>

--
Best, Jingsong Lee