Hi folks,

I was wondering if it's possible to submit register and run a script
using PigServer in MAPREDUCE mode in an asynchronous manner; compared
to how a script is executed right now whereby the program is made to
wait till the job is finished.  Although the task is still executed in
the server, I can't programmatically retrieve the job id for future
reference.
Any help is greatly appreciated.
Thanks.

There is no asynchronous API for Pig. However, Pig does have a
notification mechanism (See PigRunner.run), you can create a separate
thread to simulate the asynchronous call.

Daniel

On Fri, Feb 3, 2012 at 12:34 AM, Michael Lok <[EMAIL PROTECTED]> wrote:
> Hi folks,
>
> I was wondering if it's possible to submit register and run a script
> using PigServer in MAPREDUCE mode in an asynchronous manner; compared
> to how a script is executed right now whereby the program is made to
> wait till the job is finished.  Although the task is still executed in
> the server, I can't programmatically retrieve the job id for future
> reference.
>
>
> Any help is greatly appreciated.
>
>
> Thanks.

Not quite yet.  We're working on a web services interface to do this for Pig and other Hadoop tools, but it isn't ready for prime time just yet.  You can see the early form at https://issues.apache.org/jira/browse/HCATALOG-182

Oozie does have some support for this as well, you could check it out at http://incubator.apache.org/oozie/

Alan.

On Feb 3, 2012, at 12:34 AM, Michael Lok wrote:

> Hi folks,
>
> I was wondering if it's possible to submit register and run a script
> using PigServer in MAPREDUCE mode in an asynchronous manner; compared
> to how a script is executed right now whereby the program is made to
> wait till the job is finished.  Although the task is still executed in
> the server, I can't programmatically retrieve the job id for future
> reference.
>
>
> Any help is greatly appreciated.
>
>
> Thanks.

Hi Alan,

Thanks for the pointers. Will look at oozie for the time being :)

On Feb 11, 2012, at 2:59, Alan Gates <[EMAIL PROTECTED]> wrote:

> Not quite yet.  We're working on a web services interface to do this for Pig and other Hadoop tools, but it isn't ready for prime time just yet.  You can see the early form at https://issues.apache.org/jira/browse/HCATALOG-182
>
> Oozie does have some support for this as well, you could check it out at http://incubator.apache.org/oozie/
>
> Alan.
>
> On Feb 3, 2012, at 12:34 AM, Michael Lok wrote:
>
>> Hi folks,
>>
>> I was wondering if it's possible to submit register and run a script
>> using PigServer in MAPREDUCE mode in an asynchronous manner; compared
>> to how a script is executed right now whereby the program is made to
>> wait till the job is finished.  Although the task is still executed in
>> the server, I can't programmatically retrieve the job id for future
>> reference.
>>
>>
>> Any help is greatly appreciated.
>>
>>
>> Thanks.
>