Hi
:::::::::
QUESTION:
:::::::::
Can anyone confirm if HCatalogStore works with a hive table that was
declared with buckets?
:::::::::
DETAILS:
:::::::::
I have a table in hive that was created with buckets. But when I tried to
load the data with HCatalogStorer it fails with the following error.
Store into a partition with bucket definition from Pig/Mapreduce is not
supported.
I have a table declaration in hive:
......
PARTITIONED BY(dtStr STRING)
CLUSTERED BY(sessionid) SORTED BY(timestr) INTO 32 BUCKETS
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '1'
COLLECTION ITEMS TERMINATED BY '2'
MAP KEYS TERMINATED BY '3'
STORED AS ORC;
From pig, I load the data with HCatStorer:
STORE sessnz_all INTO '$DB.allPocData' USING
org.apache.hcatalog.pig.HCatStorer();
Details at logfile:
/home/araceli/src/bigdata/projects/cisco_webanalytics_poc/src/server/pig/scripts/pig_1386373152479.log
[araceli [ at ] greenhost03 scripts]$ pig -version
Apache Pig version 0.11.2-mapr (rexported)
compiled Aug 27 2013, 13:50:32
[araceli [ at ] greenhost03 scripts]$ hive -version
Logging initialized using configuration in
jar:file:/opt/mapr/hive/hive-0.11/lib/hive-common-0.11-mapr.jar!/hive-log4j.properties
Hive history
I have a table declaration in hive:
......
PARTITIONED BY(dtStr STRING)
CLUSTERED BY(sessionid) SORTED BY(timestr) INTO 32 BUCKETS
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '1'
COLLECTION ITEMS TERMINATED BY '2'
MAP KEYS TERMINATED BY '3'
STORED AS ORC;
From pig, I load the data with HCatStorer:
STORE sessnz_all INTO '$DB.allPocData' USING
org.apache.hcatalog.pig.HCatStorer();
Details at logfile:
/home/araceli/src/bigdata/projects/cisco_webanalytics_poc/src/server/pig/scripts/pig_1386373152479.log
[araceli [ at ] greenhost03 scripts]$ pig -version
Apache Pig version 0.11.2-mapr (rexported)
compiled Aug 27 2013, 13:50:32
[araceli [ at ] greenhost03 scripts]$ hive -version
Logging initialized using configuration in
jar:file:/opt/mapr/hive/hive-0.11/lib/hive-common-0.11-mapr.jar!/hive-log4j.properties
Hive history
:::::::::
QUESTION:
:::::::::
Can anyone confirm if HCatalogStore works with a hive table that was
declared with buckets?
:::::::::
DETAILS:
:::::::::
I have a table in hive that was created with buckets. But when I tried to
load the data with HCatalogStorer it fails with the following error.
Store into a partition with bucket definition from Pig/Mapreduce is not
supported.
I have a table declaration in hive:
......
PARTITIONED BY(dtStr STRING)
CLUSTERED BY(sessionid) SORTED BY(timestr) INTO 32 BUCKETS
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '1'
COLLECTION ITEMS TERMINATED BY '2'
MAP KEYS TERMINATED BY '3'
STORED AS ORC;
From pig, I load the data with HCatStorer:
STORE sessnz_all INTO '$DB.allPocData' USING
org.apache.hcatalog.pig.HCatStorer();
Details at logfile:
/home/araceli/src/bigdata/projects/cisco_webanalytics_poc/src/server/pig/scripts/pig_1386373152479.log
[araceli [ at ] greenhost03 scripts]$ pig -version
Apache Pig version 0.11.2-mapr (rexported)
compiled Aug 27 2013, 13:50:32
[araceli [ at ] greenhost03 scripts]$ hive -version
Logging initialized using configuration in
jar:file:/opt/mapr/hive/hive-0.11/lib/hive-common-0.11-mapr.jar!/hive-log4j.properties
Hive history
I have a table declaration in hive:
......
PARTITIONED BY(dtStr STRING)
CLUSTERED BY(sessionid) SORTED BY(timestr) INTO 32 BUCKETS
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '1'
COLLECTION ITEMS TERMINATED BY '2'
MAP KEYS TERMINATED BY '3'
STORED AS ORC;
From pig, I load the data with HCatStorer:
STORE sessnz_all INTO '$DB.allPocData' USING
org.apache.hcatalog.pig.HCatStorer();
Details at logfile:
/home/araceli/src/bigdata/projects/cisco_webanalytics_poc/src/server/pig/scripts/pig_1386373152479.log
[araceli [ at ] greenhost03 scripts]$ pig -version
Apache Pig version 0.11.2-mapr (rexported)
compiled Aug 27 2013, 13:50:32
[araceli [ at ] greenhost03 scripts]$ hive -version
Logging initialized using configuration in
jar:file:/opt/mapr/hive/hive-0.11/lib/hive-common-0.11-mapr.jar!/hive-log4j.properties
Hive history