Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> force schema with TOBAG


Copy link to this message
-
force schema with TOBAG
I have a cogroup which effectively does a full outer join of two relations. Some of the relations are blank, so I have a FOREACH statement like

grouped = COGROUP relation1 BY x, relation2 BY y;
normalized = FOREACH grouped {
   normal1 = TOBAG('$ID_NULL', 0L);
   value1 = ( IsEmpty(relation1) ? normal1 : relation1 );
   GENERATE relation1, relation2;
}

I get an error on the bincond that left and right schemas don't match. I'm informed that TOBAG return
bag{:tuple(:NULL)}
for the schema
and relation1 is
bag{:tuple(id:chararrat,timestamp:long)}

I'm running this on EMR which has a modified version of .0.9.2. Any thoughts on how to force TOBAG's schema to match relation1's?

David
+
Cheolsoo Park 2012-10-30, 18:29
+
David LaBarbera 2012-10-31, 11:53
+
Cheolsoo Park 2012-10-31, 16:23
+
David LaBarbera 2012-10-31, 16:54
+
Cheolsoo Park 2012-10-31, 17:05