Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Does Pig support HCatalogStorer table with buckets

$
0
0
Hi

:::::::::

QUESTION:

:::::::::

Can anyone confirm if HCatalogStore works with a hive table that was
declared with buckets?

:::::::::

DETAILS:

:::::::::

I have a table in hive that was created with buckets. But when I tried to
load the data with HCatalogStorer it fails with the following error.

Store into a partition with bucket definition from Pig/Mapreduce is not
supported.

I have a table declaration in hive:

......

PARTITIONED BY(dtStr STRING)

CLUSTERED BY(sessionid) SORTED BY(timestr) INTO 32 BUCKETS

ROW FORMAT DELIMITED

FIELDS TERMINATED BY '1'

COLLECTION ITEMS TERMINATED BY '2'

MAP KEYS TERMINATED BY '3'

STORED AS ORC;

From pig, I load the data with HCatStorer:

STORE sessnz_all INTO '$DB.allPocData' USING
org.apache.hcatalog.pig.HCatStorer();

Details at logfile:
/home/araceli/src/bigdata/projects/cisco_webanalytics_poc/src/server/pig/scripts/pig_1386373152479.log

[araceli [ at ] greenhost03 scripts]$ pig -version

Apache Pig version 0.11.2-mapr (rexported)

compiled Aug 27 2013, 13:50:32

[araceli [ at ] greenhost03 scripts]$ hive -version

Logging initialized using configuration in
jar:file:/opt/mapr/hive/hive-0.11/lib/hive-common-0.11-mapr.jar!/hive-log4j.properties

Hive history

I have a table declaration in hive:

......

PARTITIONED BY(dtStr STRING)

CLUSTERED BY(sessionid) SORTED BY(timestr) INTO 32 BUCKETS

ROW FORMAT DELIMITED

FIELDS TERMINATED BY '1'

COLLECTION ITEMS TERMINATED BY '2'

MAP KEYS TERMINATED BY '3'

STORED AS ORC;

From pig, I load the data with HCatStorer:

STORE sessnz_all INTO '$DB.allPocData' USING
org.apache.hcatalog.pig.HCatStorer();

Details at logfile:
/home/araceli/src/bigdata/projects/cisco_webanalytics_poc/src/server/pig/scripts/pig_1386373152479.log

[araceli [ at ] greenhost03 scripts]$ pig -version

Apache Pig version 0.11.2-mapr (rexported)

compiled Aug 27 2013, 13:50:32

[araceli [ at ] greenhost03 scripts]$ hive -version

Logging initialized using configuration in
jar:file:/opt/mapr/hive/hive-0.11/lib/hive-common-0.11-mapr.jar!/hive-log4j.properties

Hive history

Viewing all articles
Browse latest Browse all 5648

Trending Articles