Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> issue with partioning sdf


+
Aleksandr Elbakyan 2012-02-02, 01:04
Copy link to this message
-
Re: issue with partioning sdf
I think pig will use default partitioner for the same.

You can use following syntax--
A = load 'input_data';
B = group A by $0 PARTITION BY
org.apache.pig.test.utils.SimpleCustomPartitioner parallel 2;

Take a look-
https://issues.apache.org/jira/browse/PIG-282

Thanks,
Aniket

On Wed, Feb 1, 2012 at 5:04 PM, Aleksandr Elbakyan <[EMAIL PROTECTED]>wrote:

> Hello All,
>
> I am trying to understand how does pig group partitioning work, I was not
> able to find any documentation regarding what happen under the hood.
>
>
> For example
>
> B = GROUP A BY age;
>
> Does pig partition data by age? Or it will partition by something else?
>
>
> Other question:
> If I want to create custom partitioner can I pass fields I want data be
> partition by or it will be the same as group by key?
>
>
> Regards,
> Aleksandr
>
>
--
"...:::Aniket:::... Quetzalco@tl"
+
Alan Gates 2012-02-02, 16:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB