Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Should you be able to nest operators in a foreach like you normally can?


Copy link to this message
-
Re: Should you be able to nest operators in a foreach like you normally can?
Gianmarco De Francisci Mo... 2011-11-13, 10:13
I think there is no good reason to leave it like this.
I was already thinking about making a major update to Pig grammar in order
to simplify and rationalize it.
See also PIG-2138 for example.
If I manage to bring myself to work on it, this improvement should
definitely go in as well.

Could you open a Jira to keep track of the idea?

Cheers,
--
Gianmarco

On Sun, Nov 13, 2011 at 02:25, Jonathan Coveney <[EMAIL PROTECTED]> wrote:

> It's not the most important issue by far (I think the scoped names issue is
> more frustrating, as far as usability), it's just a bit counterintuitive
> that the syntax would not be consistent. What reason is there (besides
> implementation) for a user to assume that syntax inside of the foreach and
> outside of the foreach would be different (except for the nested foreach
> which is a documented technical limitation that someone is working on)?
>
> 2011/11/12 Gianmarco De Francisci Morales <[EMAIL PROTECTED]
> >
>
> > I think the answer is:
> > annoying to implement and with not so compelling use cases.
> >
> > The nested expressions take a different path in the parser in order to
> > distinguish them from the non-nested ones. So you would probably need to
> > replicate some code here and there to make this work. It can be done if
> you
> > think it is worth doing.
> >
> > Cheers,
> > --
> > Gianmarco
> >
> >
> >
> > On Fri, Nov 11, 2011 at 21:22, Jonathan Coveney <[EMAIL PROTECTED]>
> > wrote:
> >
> > > This works:
> > >
> > > a = load 'thing';
> > > b = foreach (group a all) {
> > >  sorted = order a by $0;
> > >  top = limit sorted 100;
> > >  generate flatten(top);
> > > }
> > >
> > > This does not (neither in pig8 nor in pig9, to my knowledge):
> > >
> > > a = load 'thing';
> > > b = foreach (group a all) {
> > >  top = limit (order a by $0) 100;
> > >  generate flatten(top);
> > > }
> > >
> > > Just curious if this is a design choice, design limitation, annoying to
> > > implement, etc. Thought I'd bring it up.
> > >
> > > Jon, your man for "nested foreach" posts to the dev listserv
> > >
> >
>