Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Pig GROUP operator - Data is shuffled and wind up together for the same grouping key


Copy link to this message
-
Re: Pig GROUP operator - Data is shuffled and wind up together for the same grouping key
Hi Viswa,

All records with the same key ending up in the same reducer is expected.
Can you provide us with your script and a sample input/output if you are
seeing something different?

On Thursday, August 29, 2013, Viswanathan J wrote:

> Hi,
>
> I'm using pig version 0.11.0
>
> While using GROUP operator in Pig all the data is shuffled, so that rows in
> different partitions that have the same grouping key wind up together and
> got wrong results for grouping.
>
> While storing the result data, it is share work between multiple
> calculations.
>
> How to solve this? Please advice.
>
> --
> Regards,
> Viswa.J
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB