Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Nested Binary Condition operators takes too much time to compile, PIG 0.11

$
0
0
Hello,

Below is the script which is working fine on PIG 0.8. Currently I am moving
from PIG 0.8 to 0.11. Trying to execute the same code on PIG 0.11. Its
taking too much time to compile the script.

A = LOAD 'data' USING PigStorage(',') AS
(code1:chararray,code2:chararray,code3:chararray);
B = FOREACH A GENERATE
(code1 MATCHES '(val1|val2|val3|val4|val5|val6|val7|val8|val9)'?
code1:'oth') AS code1,
(code2 MATCHES '(c1|c2|c3|v4|c5)'? code2: 'oth') AS code2,
code3;
C = FOREACH B GENERATE
(code2 == 'c1'? (code1 == 'val1' AND code3 =='xval' ? 1 : ( code1
== 'val1'? 2: ( code1 == 'val2'? 3: ( code1 == 'val3'? 4: ( code1 ==
'val4'? 5:( code1 == 'val5'? 6:( code1 == 'val6'? 7 : ( code1 == 'val7'? 8:
( code1 == 'val8'? 9 :10))))))))) :
(code2 == 'c2' ? (code1 == 'val1' AND code3 =='xval' ? 11 : ( code1
== 'val1'? 12: ( code1 == 'val2'? 13: ( code1 == 'val3'? 14: ( code1 ==
'val4'? 15:( code1 == 'val5'? 16:( code1 == 'val6'? 17 : 18))))))) :
(code2 == 'c3' ? (code1 =='val1' ? 19 : ( code1 == 'val2'? 20: (
code1 == 'val3'? 21: ( code1 == 'val4'? 22: ( code1 == 'val5'? 23:24))))):
(code2 == 'c4' ? (code1 =='val1' ? 25: ( code1 == 'val2'? 26: (
code1 == 'val3'? 27: ( code1 == 'val4'? 28: 29)))) :
(code1 =='val1' ? 30 : ( code1 == 'val2'? 31: ( code1 ==
'val3'? 32: ( code1 == 'val4'? 33: ( code1 == 'val5'? 34:35))))))))) as
code1,
(code2 == 'c1' ? 1: (code2== 'c2' ? 2 :( code2=='c3' ? 3 : (code2==
'c4'? 4:5)))) AS code2;

I am using Cloudera distribution. I tried same code on grunts for PIG 0.8
and PIG.11. On PIG 0.8 it compiles within a moment. On PIG 0.11 I executed
the line and waited almost for 12 mins, but control didn't come back grunt.

Do I need to set anything which I missed and having this problem?

Viewing all articles
Browse latest Browse all 5648

Trending Articles