Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Pig GROUP operator - Data is shuffled and wind up together for the same grouping key


+
Viswanathan J 2013-08-29, 08:30
Copy link to this message
-
Re: Pig GROUP operator - Data is shuffled and wind up together for the same grouping key
Hi Viswa,

All records with the same key ending up in the same reducer is expected.
Can you provide us with your script and a sample input/output if you are
seeing something different?

On Thursday, August 29, 2013, Viswanathan J wrote:

> Hi,
>
> I'm using pig version 0.11.0
>
> While using GROUP operator in Pig all the data is shuffled, so that rows in
> different partitions that have the same grouping key wind up together and
> got wrong results for grouping.
>
> While storing the result data, it is share work between multiple
> calculations.
>
> How to solve this? Please advice.
>
> --
> Regards,
> Viswa.J
>