Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> force schema with TOBAG


Copy link to this message
-
force schema with TOBAG
I have a cogroup which effectively does a full outer join of two relations. Some of the relations are blank, so I have a FOREACH statement like

grouped = COGROUP relation1 BY x, relation2 BY y;
normalized = FOREACH grouped {
   normal1 = TOBAG('$ID_NULL', 0L);
   value1 = ( IsEmpty(relation1) ? normal1 : relation1 );
   GENERATE relation1, relation2;
}

I get an error on the bincond that left and right schemas don't match. I'm informed that TOBAG return
bag{:tuple(:NULL)}
for the schema
and relation1 is
bag{:tuple(id:chararrat,timestamp:long)}

I'm running this on EMR which has a modified version of .0.9.2. Any thoughts on how to force TOBAG's schema to match relation1's?

David
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB