Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Sometimes-Algebraic functions

Copy link to this message
Re: Sometimes-Algebraic functions
Yup, that makes sense. Class returned by argToFuncMapping needs to implement
Algebraic instead of EvalFunc itself.

On Mon, Aug 29, 2011 at 15:46, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Had a fun discovery today: a user tried to do the following (perfectly
> reasonable) thing:
> bag_sizes = foreach grouped_data genrate
>  group, SIZE(grouped_data);
> .. and it was excessively slow, because SIZE is not algebraic.
> I had him switch to COUNT_STAR, and that worked, of course.
> But he reasonably pointed out that SIZE should work, too. It's documented
> to
> work on bags, after all.
> I tried switching SIZE's argToFunc mapping to return
> COUNT_STAR.class.getName(), but discovered much to my dismay that algebraic
> optimization still does not get invoked.
> Presumably because we check if SIZE is instanceof Algebraic, instead of
> checking the class that's returned by argToFuncMapping.
> Is that about right? Do you guys agree that's a bug?
> D