Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Get field from bag with constraints from same relation


+
Thomas Bach 2013-01-22, 11:55
+
Thomas Bach 2013-01-22, 16:24
+
Cheolsoo Park 2013-01-22, 19:31
Copy link to this message
-
Re: Get field from bag with constraints from same relation
Thomas Bach 2013-01-23, 14:32
On Tue, Jan 22, 2013 at 11:31:23AM -0800, Cheolsoo Park wrote:
>
> Try this:
>
> data1 = LOAD '1.txt' USING PigStorage('|') AS (n:int,
> B:bag{(m:int,s:chararray)});
> data2 = FOREACH data1 GENERATE n, FLATTEN(B);
> data3 = FILTER data2 BY B::m <= n;
> data4 = GROUP data3 BY n;
> data5 = FOREACH data4 {
>     data6 = ORDER data3 BY B::m DESC;
>     data7 = LIMIT data6 1;
>     GENERATE data7;
> }
> data8 = FOREACH data5 GENERATE FLATTEN(data7);
> data9 = FOREACH data8 GENERATE n, B::s;
> DUMP data9;
>
> The input is:
> 4|{(1,abc),(2,cde),(5,efg)}
> 2|{(1,foo),(2,bar),(5,baz)}
> 7|{(1,bounce),(2,frotz),(5,trotz)}
>
> The output is:
> (2,bar)
> (4,cde)
> (7,trotz)

It's much more complicated than I thought. :/

But, it works like a charm. Thank you! :)

Regards,
Thomas Bach.