Hello,

I'm looking for some feedback on how real world pig jobs are run in production, how are they managed and scheduled etc...?  We are currently using bash scripts and cron to fire jobs off and we wondering what real world experiences have revealed about how to build a robust system that uses scheduled jobs with dependencies on each other etc...

Thanks for any info,

Cheers!

Hi Corbin,

Recently, LinkedIn opensourced their own scheduler, 'Azkaban'.
See the announcement,
http://mail-archives.apache.org/mod_mbox/hadoop-general/201003.mbox/%3CC7D7BBAA.3E97%[EMAIL PROTECTED]%3E

Regards,
-Youngwoo
2010/4/1 Corbin Hoenes <[EMAIL PROTECTED]>

> Hello,
>
> I'm looking for some feedback on how real world pig jobs are run in
> production, how are they managed and scheduled etc...?  We are currently
> using bash scripts and cron to fire jobs off and we wondering what real
> world experiences have revealed about how to build a robust system that uses
> scheduled jobs with dependencies on each other etc...
>
> Thanks for any info,
>
> Cheers!