Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> STORE with variable?


Copy link to this message
-
STORE with variable?
Is there a way to use STORE with variable or some other way to achieve what
I need.

I have something like this:

grunt> DESCRIBE A;
A: {f1, f2, f3, ...}

grunt> DUMP A;
(v1, x2, x3, ...)
(v2, x4, x5, ...)
(v1, x6, x6, ...)
...

I do so processing and then group by f1 and would like to save the result in
different directories for different f1, like this:

     /result/f1/result_for_v1
     /result/f2/result_for_v2
     /result/f2/result_for_v2
     ...

I know I could use SPLIT, but I have 100+ unique values for f1, and number
of uniques varies each time I process. It will be nice I don't have list 100
BY lines with SPLIT and I certainly do not want to maintain the list of
possible values for f1 in my Pig script.

Thanks!
Dexin
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB