Apache Flink 中文用户邮件列表

image

classic Classic list List threaded Threaded
♦
♦
 4 messages Options
张颖
Reply | Threaded
Open this post in threaded view
♦
♦
|

image

张颖
hello, I meet a problem just like this: My flink version is 1.12.0，and I change the version to 1.12.3, the problem still exist.

I create two tables which have data type map, I want to union two tables to one, but when i run on sql-client, I met such a problem:(number_feature is a map data)



but when I run this sql,it has correct result:"(SELECT query FROM map_string_string1) UNION (SELECT query FROM map_string_string2)"
The reason why it cannot support map union is a problem?

this is my create table statements:
CREATE TABLE `map_string_string1`(
  `query` string,
  `wid` string,
  `index` int,
  `page` string,
  `hc_cid1` string,
  `hc_cid2` string,
  `hc_cid3` string,
  `cid1` string,
  `cid2` string,
  `cid3` string,
  `ts` bigint,
  `number_feature` map<string,string>)
PARTITIONED BY (
  `dt` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'

CREATE TABLE `map_string_string2`(
  `query` string,
  `wid` string,
  `index` int,
  `page` string,
  `hc_cid1` string,
  `hc_cid2` string,
  `hc_cid3` string,
  `cid1` string,
  `cid2` string,
  `cid3` string,
  `ts` bigint,
  `number_feature` map<string,string>)
PARTITIONED BY (
  `dt` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'



 
张颖
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re:image

张颖






At 2021-03-29 17:42:05, "张颖" <[hidden email]> wrote:

hello, I meet a problem just like this: My flink version is 1.12.0，and I change the version to 1.12.3, the problem still exist.

I create two tables which have data type map, I want to union two tables to one, but when i run on sql-client, I met such a problem:(number_feature is a map data)



but when I run this sql,it has correct result:"(SELECT query FROM map_string_string1) UNION (SELECT query FROM map_string_string2)"
The reason why it cannot support map union is a problem?

this is my create table statements:
CREATE TABLE `map_string_string1`(
  `query` string,
  `wid` string,
  `index` int,
  `page` string,
  `hc_cid1` string,
  `hc_cid2` string,
  `hc_cid3` string,
  `cid1` string,
  `cid2` string,
  `cid3` string,
  `ts` bigint,
  `number_feature` map<string,string>)
PARTITIONED BY (
  `dt` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'

CREATE TABLE `map_string_string2`(
  `query` string,
  `wid` string,
  `index` int,
  `page` string,
  `hc_cid1` string,
  `hc_cid2` string,
  `hc_cid3` string,
  `cid1` string,
  `cid2` string,
  `cid3` string,
  `ts` bigint,
  `number_feature` map<string,string>)
PARTITIONED BY (
  `dt` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'



 



 
张颖
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re:image

张颖
In reply to this post by 张颖
this is the image:






At 2021-03-29 17:42:05, "张颖" <[hidden email]> wrote:

hello, I meet a problem just like this: My flink version is 1.12.0，and I change the version to 1.12.3, the problem still exist.

I create two tables which have data type map, I want to union two tables to one, but when i run on sql-client, I met such a problem:(number_feature is a map data)



but when I run this sql,it has correct result:"(SELECT query FROM map_string_string1) UNION (SELECT query FROM map_string_string2)"
The reason why it cannot support map union is a problem?

this is my create table statements:
CREATE TABLE `map_string_string1`(
  `query` string,
  `wid` string,
  `index` int,
  `page` string,
  `hc_cid1` string,
  `hc_cid2` string,
  `hc_cid3` string,
  `cid1` string,
  `cid2` string,
  `cid3` string,
  `ts` bigint,
  `number_feature` map<string,string>)
PARTITIONED BY (
  `dt` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'

CREATE TABLE `map_string_string2`(
  `query` string,
  `wid` string,
  `index` int,
  `page` string,
  `hc_cid1` string,
  `hc_cid2` string,
  `hc_cid3` string,
  `cid1` string,
  `cid2` string,
  `cid3` string,
  `ts` bigint,
  `number_feature` map<string,string>)
PARTITIONED BY (
  `dt` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'



 



 
张颖
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re:image

张颖
In reply to this post by 张颖
this is the contents of this image:


Flink SQL> (SELECT number_feature FROM map_string_string1) UNION (SELECT number_feature FROM map_string_string2);


2021-03-29 16:57:15,479 INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient         [] - Trying to connect to metastore with URI thrift://localhost:9083
2021-03-29 16:57:15,483 INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient         [] - Opened a connection to metastore, current connections: 2
2021-03-29 16:57:15,530 INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient         [] - Connected to metastore.
2021-03-29 16:57:15,530 INFO  org.apache.hadoop.hive.metastore.RetryingMetaStoreClient     [] - RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.metastore.HiveMetaStoreClient ugi=zhangying480 (auth:SIMPLE) retries=1 delay=1 lifetime=0
2021-03-29 16:57:15,580 INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient         [] - Closed a connection to metastore, current connections: 1
2021-03-29 16:57:15,651 INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient         [] - Trying to connect to metastore with URI thrift://localhost:9083
2021-03-29 16:57:15,652 INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient         [] - Opened a connection to metastore, current connections: 2
2021-03-29 16:57:15,652 INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient         [] - Connected to metastore.
2021-03-29 16:57:15,652 INFO  org.apache.hadoop.hive.metastore.RetryingMetaStoreClient     [] - RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.metastore.HiveMetaStoreClient ugi=zhangying480 (auth:SIMPLE) retries=1 delay=1 lifetime=0
2021-03-29 16:57:15,698 INFO  org.apache.hadoop.hive.metastore.HiveMetaStoreClient         [] - Closed a connection to metastore, current connections: 1




[ERROR] Could not execute SQL statement. Reason:
java.lang.UnsupportedOperationException: Unsupported type: MAP<STRING, STRING>






























At 2021-03-29 17:42:05, "张颖" <[hidden email]> wrote:

hello, I meet a problem just like this: My flink version is 1.12.0，and I change the version to 1.12.3, the problem still exist.


I create two tables which have data type map, I want to union two tables to one, but when i run on sql-client, I met such a problem:(number_feature is a map data)






but when I run this sql,it has correct result:"(SELECT query FROM map_string_string1) UNION (SELECT query FROM map_string_string2)"
The reason why it cannot support map union is a problem?


this is my create table statements:
CREATE TABLE `map_string_string1`(
  `query` string,
  `wid` string,
  `index` int,
  `page` string,
  `hc_cid1` string,
  `hc_cid2` string,
  `hc_cid3` string,
  `cid1` string,
  `cid2` string,
  `cid3` string,
  `ts` bigint,
  `number_feature` map<string,string>)
PARTITIONED BY (
  `dt` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'


CREATE TABLE `map_string_string2`(
  `query` string,
  `wid` string,
  `index` int,
  `page` string,
  `hc_cid1` string,
  `hc_cid2` string,
  `hc_cid3` string,
  `cid1` string,
  `cid2` string,
  `cid3` string,
  `ts` bigint,
  `number_feature` map<string,string>)
PARTITIONED BY (
  `dt` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t'
  LINES TERMINATED BY '\n'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'






 
Free forum by Nabble Edit this page