Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Pig GROUP operator - Data is shuffled and wind up together for the same grouping key


Copy link to this message
-
Re: Pig GROUP operator - Data is shuffled and wind up together for the same grouping key
Prashant Kommireddi 2013-08-29, 17:02
Hi Viswa,

All records with the same key ending up in the same reducer is expected.
Can you provide us with your script and a sample input/output if you are
seeing something different?

On Thursday, August 29, 2013, Viswanathan J wrote:

> Hi,
>
> I'm using pig version 0.11.0
>
> While using GROUP operator in Pig all the data is shuffled, so that rows in
> different partitions that have the same grouping key wind up together and
> got wrong results for grouping.
>
> While storing the result data, it is share work between multiple
> calculations.
>
> How to solve this? Please advice.
>
> --
> Regards,
> Viswa.J
>