Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Changing the schema before Storing


Copy link to this message
-
Changing the schema before Storing
yaboulna@... 2012-12-11, 03:31
Hello,

I'm using HBaseStorage and I want to change the layout of the schema  
before storage. Specifically I want to group some values into a tuple  
(thus reducing the number of repetitions of the row and column keys).

Even though the JavaDoc gives an example that uses AS schema Grunt  
complains that it is not parsable. Here's what I am trying:

STORE dataToStore INTO 'hbase://tableName' USING  
HBaseStorage('cf:tuple, cf:date') AS TOTUPLE(val1, val2, val3), date;

Is this possible? Or do I have to do the transformation in a separate step:

dataTransformed = FOREACH dataToStore GENERATE TOTUPLE(val1, val2,  
val3), date;

In case of the latter, can Pig be told to merge this step with the  
next one? I tried a nested FOREACH where I can have an assignment  
operation, but I quickly found out that STORE is not supported within  
the FOREACH.. what was I thinking :).

Thanks!

-- Younos