Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Using PARALLEL clause for nested operations


Copy link to this message
-
RE: Using PARALLEL clause for nested operations
Santhosh Srinivasan 2008-05-21, 17:57
It's a syntactically correct statement in QueryParser. However, the pig
statement is shipped to QueryParser via Grunt which does not expect the
PARALLEL keyword. There are two parsers you are dealing with:

1. Grunt parser that ships pig statements to QueryParser
2. QueryParser that parses the pig statements

The problem in this specific case is with the former and not the latter.

Santhosh

-----Original Message-----
From: pi song [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, May 21, 2008 7:59 AM
To: [EMAIL PROTECTED]
Subject: Re: Using PARALLEL clause for nested operations

Of course, this is an open source project. Everybody can see the source
code.

Have a look at
http://svn.apache.org/viewvc/incubator/pig/trunk/src/org/apache/pig/impl
/logicalLayer/parser/QueryParser.jjt?view=log

And here is the root source location:
http://svn.apache.org/viewvc/incubator/pig/trunk

It's a JavaCC file so a bit difficult to read. No good debugging tool
either.

Pi
On Thu, May 22, 2008 at 12:53 AM, Cosmin Lehene <[EMAIL PROTECTED]>
wrote:

> Yes, well, I tried both. Didn't see the grammar definition, though.
Where
> can I find it, so I could look in the code as well?
>
> Thanks,
> Cosmin
>
>
> On 5/21/08 5:50 PM, "pi song" <[EMAIL PROTECTED]> wrote:
>
> > According to the grammar file, it should be:-
> >
> > FOREACH X {
> >    ...
> >    ... GENERATE ... ;
> > } PARALLLEL N;
> >
> > But it doesn't work. I guess this is a bug!!
> >
> > Pi
> >
> > On Thu, May 22, 2008 at 12:39 AM, Cosmin Lehene <[EMAIL PROTECTED]>
> wrote:
> >
> >> Hi,
> >>
> >> I'm trying to use the PARALLEL clause inside a
> >>
> >> FOREACH X {
> >>    ...
> >>    ... GENERATE ... PARALLLEL N;
> >> }
> >>
> >> And it fails.
> >>
> >> 670  [main] ERROR org.apache.pig.tools.grunt.Grunt  - Encountered
> >> "PARALLEL"
> >> at line 3, column 64.
> >> Was expecting one of:
> >>    ";" ...
> >>    "," ...
> >>    ":" ...
> >>
> >> Is it limited to regular statements?
> >>
> >> Thanks,
> >> Cosmin
> >>
> >>
>
>