Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> DISTINCT and paritioner


+
William Oberman 2013-07-17, 18:27
+
William Oberman 2013-07-17, 18:30
+
Alan Gates 2013-07-18, 15:28
Copy link to this message
-
Re: DISTINCT and paritioner
Thanks for letting me know!

https://issues.apache.org/jira/browse/PIG-3385
On Thu, Jul 18, 2013 at 11:28 AM, Alan Gates <[EMAIL PROTECTED]> wrote:

> You're correct.  It looks like an optimization was put in to make distinct
> use a special partitioner which prevents the user from setting the
> partitioner.  Could you file a JIRA against the docs so we can get that
> fixed?
>
> Alan.
>
> On Jul 17, 2013, at 11:27 AM, William Oberman wrote:
>
> > The docs say DISTINCT can take a custom partitioner.  How does that work?
> > What is "K" and "V"?
> > I'm having some doubts the docs are correct.  I wrote a test partitioner
> > that does a System.out of K and V.  I then wrote simple scripts to do
> JOIN,
> > GROUP and DISTINCT.  For JOIN and GROUP I see my system.outs(*).  For
> > DISTINCT, I see nothing....
> >
> > Using 0.11.1.
> >
> > will
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB