Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Sometimes-Algebraic functions


+
Dmitriy Ryaboy 2011-08-29, 22:46
Copy link to this message
-
Re: Sometimes-Algebraic functions
Yup, that makes sense. Class returned by argToFuncMapping needs to implement
Algebraic instead of EvalFunc itself.

Ashutosh
On Mon, Aug 29, 2011 at 15:46, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Had a fun discovery today: a user tried to do the following (perfectly
> reasonable) thing:
>
> bag_sizes = foreach grouped_data genrate
>  group, SIZE(grouped_data);
>
> .. and it was excessively slow, because SIZE is not algebraic.
>
> I had him switch to COUNT_STAR, and that worked, of course.
>
> But he reasonably pointed out that SIZE should work, too. It's documented
> to
> work on bags, after all.
>
> I tried switching SIZE's argToFunc mapping to return
> COUNT_STAR.class.getName(), but discovered much to my dismay that algebraic
> optimization still does not get invoked.
>
> Presumably because we check if SIZE is instanceof Algebraic, instead of
> checking the class that's returned by argToFuncMapping.
>
> Is that about right? Do you guys agree that's a bug?
>
> D
>
+
Dmitriy Ryaboy 2011-08-30, 21:24