Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - How to automate running pig scripts


Copy link to this message
-
Re: How to automate running pig scripts
Prashant Kommireddi 2012-05-26, 18:19
Take a look at Oozie or Azkaban.

http://incubator.apache.org/oozie/

I prefer Oozie, its a lot more comprehensive. AZK does not seem to be fully
baked (though installation and getting started could be easier).

On Sat, May 26, 2012 at 11:09 AM, Juan Martin Pampliega <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> I need to have pig scripts run automatically every certain amount of time.
> Also, I need to know if a script is still running so that the next one does
> not start until the previous one has finished and also I need to register
> somewhere the end state of each script run to display it in a table on a
> web page.
> My last requirement is to have the scripts receive some parameters that
> will be from which id to start processing. So for example the second time
> the script is run it should know which was the last id that was processed
> and start from there onwards.
>
> Does anyone have any suggestions on how to approach this?
>
> I was thinking of using Jython to run the Pig scripts programmatically but
> I'm having trouble calling all these from Java and I'm always getting
> success status even when the pig script fails.
>
> Any other suggestions?
>