-Re: How to automate running pig scripts
Prashant Kommireddi 2012-05-26, 18:19
Take a look at Oozie or Azkaban.
I prefer Oozie, its a lot more comprehensive. AZK does not seem to be fully
baked (though installation and getting started could be easier).
On Sat, May 26, 2012 at 11:09 AM, Juan Martin Pampliega <
[EMAIL PROTECTED]> wrote:
> I need to have pig scripts run automatically every certain amount of time.
> Also, I need to know if a script is still running so that the next one does
> not start until the previous one has finished and also I need to register
> somewhere the end state of each script run to display it in a table on a
> web page.
> My last requirement is to have the scripts receive some parameters that
> will be from which id to start processing. So for example the second time
> the script is run it should know which was the last id that was processed
> and start from there onwards.
> Does anyone have any suggestions on how to approach this?
> I was thinking of using Jython to run the Pig scripts programmatically but
> I'm having trouble calling all these from Java and I'm always getting
> success status even when the pig script fails.
> Any other suggestions?