Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Sometimes-Algebraic functions


+
Dmitriy Ryaboy 2011-08-29, 22:46
Copy link to this message
-
Re: Sometimes-Algebraic functions
Yup, that makes sense. Class returned by argToFuncMapping needs to implement
Algebraic instead of EvalFunc itself.

Ashutosh
On Mon, Aug 29, 2011 at 15:46, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Had a fun discovery today: a user tried to do the following (perfectly
> reasonable) thing:
>
> bag_sizes = foreach grouped_data genrate
>  group, SIZE(grouped_data);
>
> .. and it was excessively slow, because SIZE is not algebraic.
>
> I had him switch to COUNT_STAR, and that worked, of course.
>
> But he reasonably pointed out that SIZE should work, too. It's documented
> to
> work on bags, after all.
>
> I tried switching SIZE's argToFunc mapping to return
> COUNT_STAR.class.getName(), but discovered much to my dismay that algebraic
> optimization still does not get invoked.
>
> Presumably because we check if SIZE is instanceof Algebraic, instead of
> checking the class that's returned by argToFuncMapping.
>
> Is that about right? Do you guys agree that's a bug?
>
> D
>
+
Dmitriy Ryaboy 2011-08-30, 21:24
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB