let's say my pig script generates 2 MR jobs.

it seems that currently pig parser won't try to parse the second part until
it finishes running the first MR.

by that time 1 hour may have passed and now pig says:

blahblah does not exists in the schema of myvar_name : { x:int ,
y:chararray.......}

so a lot of time is wasted, particularly in debugging.

is there a mode like pig -c
(like perl -c ) to check syntax errors?

Thanks
Yang

Pig SHOULD parse the whole script, AFAIK. There are certain errors that
will only surface at runtime, but in general, parsing errors should be
surfacing early. Do you happen to have an example?

2012/6/28 Yang <[EMAIL PROTECTED]>

> let's say my pig script generates 2 MR jobs.
>
> it seems that currently pig parser won't try to parse the second part until
> it finishes running the first MR.
>
> by that time 1 hour may have passed and now pig says:
>
> blahblah does not exists in the schema of myvar_name : { x:int ,
> y:chararray.......}
>
> so a lot of time is wasted, particularly in debugging.
>
> is there a mode like pig -c
> (like perl -c ) to check syntax errors?
>
> Thanks
> Yang
>

thanks
it was simply "blahblah field does not existing in schema for my_var :
{......} "
On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <[EMAIL PROTECTED]>wrote:

> Pig SHOULD parse the whole script, AFAIK. There are certain errors that
> will only surface at runtime, but in general, parsing errors should be
> surfacing early. Do you happen to have an example?
>
> 2012/6/28 Yang <[EMAIL PROTECTED]>
>
> > let's say my pig script generates 2 MR jobs.
> >
> > it seems that currently pig parser won't try to parse the second part
> until
> > it finishes running the first MR.
> >
> > by that time 1 hour may have passed and now pig says:
> >
> > blahblah does not exists in the schema of myvar_name : { x:int ,
> > y:chararray.......}
> >
> > so a lot of time is wasted, particularly in debugging.
> >
> > is there a mode like pig -c
> > (like perl -c ) to check syntax errors?
> >
> > Thanks
> > Yang
> >
>

Do you have an example?

2012/6/28 Yang <[EMAIL PROTECTED]>

> thanks
>
>
> it was simply "blahblah field does not existing in schema for my_var :
> {......} "
>
>
> On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <[EMAIL PROTECTED]
> >wrote:
>
> > Pig SHOULD parse the whole script, AFAIK. There are certain errors that
> > will only surface at runtime, but in general, parsing errors should be
> > surfacing early. Do you happen to have an example?
> >
> > 2012/6/28 Yang <[EMAIL PROTECTED]>
> >
> > > let's say my pig script generates 2 MR jobs.
> > >
> > > it seems that currently pig parser won't try to parse the second part
> > until
> > > it finishes running the first MR.
> > >
> > > by that time 1 hour may have passed and now pig says:
> > >
> > > blahblah does not exists in the schema of myvar_name : { x:int ,
> > > y:chararray.......}
> > >
> > > so a lot of time is wasted, particularly in debugging.
> > >
> > > is there a mode like pig -c
> > > (like perl -c ) to check syntax errors?
> > >
> > > Thanks
> > > Yang
> > >
> >
>

Hey Yang,

For debugging you may want the local mode, try
pig -x local

Also there are some useful commands like, DESCRIBE, ILLUSTRATE

Ruslan

On Fri, Jun 29, 2012 at 7:38 AM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
> Do you have an example?
>
> 2012/6/28 Yang <[EMAIL PROTECTED]>
>
>> thanks
>>
>>
>> it was simply "blahblah field does not existing in schema for my_var :
>> {......} "
>>
>>
>> On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Pig SHOULD parse the whole script, AFAIK. There are certain errors that
>> > will only surface at runtime, but in general, parsing errors should be
>> > surfacing early. Do you happen to have an example?
>> >
>> > 2012/6/28 Yang <[EMAIL PROTECTED]>
>> >
>> > > let's say my pig script generates 2 MR jobs.
>> > >
>> > > it seems that currently pig parser won't try to parse the second part
>> > until
>> > > it finishes running the first MR.
>> > >
>> > > by that time 1 hour may have passed and now pig says:
>> > >
>> > > blahblah does not exists in the schema of myvar_name : { x:int ,
>> > > y:chararray.......}
>> > >
>> > > so a lot of time is wasted, particularly in debugging.
>> > >
>> > > is there a mode like pig -c
>> > > (like perl -c ) to check syntax errors?
>> > >
>> > > Thanks
>> > > Yang
>> > >
>> >
>>

--
Best Regards,
Ruslan Al-Fakikh

Pig does have a "-c" to check the syntax:

pig -x local -c -f x.pig

Jie

On Fri, Jun 29, 2012 at 5:02 AM, Ruslan Al-Fakikh
<[EMAIL PROTECTED]> wrote:
> Hey Yang,
>
> For debugging you may want the local mode, try
> pig -x local
>
> Also there are some useful commands like, DESCRIBE, ILLUSTRATE
>
> Ruslan
>
> On Fri, Jun 29, 2012 at 7:38 AM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
>> Do you have an example?
>>
>> 2012/6/28 Yang <[EMAIL PROTECTED]>
>>
>>> thanks
>>>
>>>
>>> it was simply "blahblah field does not existing in schema for my_var :
>>> {......} "
>>>
>>>
>>> On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <[EMAIL PROTECTED]
>>> >wrote:
>>>
>>> > Pig SHOULD parse the whole script, AFAIK. There are certain errors that
>>> > will only surface at runtime, but in general, parsing errors should be
>>> > surfacing early. Do you happen to have an example?
>>> >
>>> > 2012/6/28 Yang <[EMAIL PROTECTED]>
>>> >
>>> > > let's say my pig script generates 2 MR jobs.
>>> > >
>>> > > it seems that currently pig parser won't try to parse the second part
>>> > until
>>> > > it finishes running the first MR.
>>> > >
>>> > > by that time 1 hour may have passed and now pig says:
>>> > >
>>> > > blahblah does not exists in the schema of myvar_name : { x:int ,
>>> > > y:chararray.......}
>>> > >
>>> > > so a lot of time is wasted, particularly in debugging.
>>> > >
>>> > > is there a mode like pig -c
>>> > > (like perl -c ) to check syntax errors?
>>> > >
>>> > > Thanks
>>> > > Yang
>>> > >
>>> >
>>>
>
>
>
> --
> Best Regards,
> Ruslan Al-Fakikh
>

perfect, thanks

On Fri, Jun 29, 2012 at 10:11 AM, Jie Li <[EMAIL PROTECTED]> wrote:

> Pig does have a "-c" to check the syntax:
>
> pig -x local -c -f x.pig
>
> Jie
>
> On Fri, Jun 29, 2012 at 5:02 AM, Ruslan Al-Fakikh
> <[EMAIL PROTECTED]> wrote:
> > Hey Yang,
> >
> > For debugging you may want the local mode, try
> > pig -x local
> >
> > Also there are some useful commands like, DESCRIBE, ILLUSTRATE
> >
> > Ruslan
> >
> > On Fri, Jun 29, 2012 at 7:38 AM, Jonathan Coveney <[EMAIL PROTECTED]>
> wrote:
> >> Do you have an example?
> >>
> >> 2012/6/28 Yang <[EMAIL PROTECTED]>
> >>
> >>> thanks
> >>>
> >>>
> >>> it was simply "blahblah field does not existing in schema for my_var :
> >>> {......} "
> >>>
> >>>
> >>> On Thu, Jun 28, 2012 at 8:24 PM, Jonathan Coveney <[EMAIL PROTECTED]
> >>> >wrote:
> >>>
> >>> > Pig SHOULD parse the whole script, AFAIK. There are certain errors
> that
> >>> > will only surface at runtime, but in general, parsing errors should
> be
> >>> > surfacing early. Do you happen to have an example?
> >>> >
> >>> > 2012/6/28 Yang <[EMAIL PROTECTED]>
> >>> >
> >>> > > let's say my pig script generates 2 MR jobs.
> >>> > >
> >>> > > it seems that currently pig parser won't try to parse the second
> part
> >>> > until
> >>> > > it finishes running the first MR.
> >>> > >
> >>> > > by that time 1 hour may have passed and now pig says:
> >>> > >
> >>> > > blahblah does not exists in the schema of myvar_name : { x:int ,
> >>> > > y:chararray.......}
> >>> > >
> >>> > > so a lot of time is wasted, particularly in debugging.
> >>> > >
> >>> > > is there a mode like pig -c
> >>> > > (like perl -c ) to check syntax errors?
> >>> > >
> >>> > > Thanks
> >>> > > Yang
> >>> > >
> >>> >
> >>>
> >
> >
> >
> > --
> > Best Regards,
> > Ruslan Al-Fakikh
> >
>