Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> A question regarding schema


Copy link to this message
-
A question regarding schema
Hi All,I have a UDF that returns a tuple. The number of elements in the tuple will differ for each user. For example:
(userid1, item1, item2, item 100, item 400)(userid1, item1, item200)(userid1, item1, item2, item 100, item200, item250, item300, item 400)(userid1, item 100, item 200, item250, item300, item380, item400, item450, item480, item560, item800, item1000)
etc.
Pig script:
A = LOAD '/scratch/input.seq' USING $SEQFILE_LOADER ( '-c $TEXT_CONVERTER', '-c $TEXT_CONVERTER') AS (key: chararray, value: chararray);
UserItemAssoc = FOREACH A GENERATE myparser.myUDF(key, value) AS {(userid: chararray, itemtid: How to specify this???)};
If I want to specify the schema in the AS clause, how do I do it since the number of fields will differ in each row? Is it possible to somehow do this dynamically?
     
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB