Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> manipulating HBaseStorage map outside of a UDF?


Copy link to this message
-
manipulating HBaseStorage map outside of a UDF?
I'm using HBaseStorage to load a large column family (many columns)
into a relation, which generates a map[] on each row.  The maps are
wide and sparse (only a few keys exist on each row), and I'd ideally
like to GROUP all maps together by similar columns before passing off
to a UDF for further processing.

Is this possible?  I'd be fine with converting to bags first, but
seems TOBAG() just adds the extra bagging layer on top of a map.

Failing that, is there any manipulation I can make on these types of
relations in Pig in the case where I don't want to explicitly specify
each map key?

Norbert
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB