-Re: Oozie vs. YARN "application"
Keith Wiley 2012-05-18, 00:41
Just to be clear yarn does offer something described as dag management, but perhaps you mean it doesn't offer a dag *of* applications, only of mr jobs *within* an application. Is that what you mean?
Sent from my phone, please excuse my brevity.
Keith Wiley, [EMAIL PROTECTED], http://keithwiley.com
Arun C Murthy <[EMAIL PROTECTED]> wrote:
YARN doesn't yet have a DAGApplicationMaster which can handle a DAG of jobs.
Conceivable Oozie could offload the DAG management once it's available.
OTOH, Oozie provides much more than just DAG management - it provides time & data-availability based scheduling of workflows.
On May 17, 2012, at 4:47 PM, Keith Wiley wrote:
> Hadoop 0.23 (MRv2 or YARN) provides the concept of an "application", which is described as either a classic single MR job *or* a DAG of such jobs...which at a glance appears redundant with Oozie's primary purpose.
> How do they differ? Does 0.23 effectively obviate the service that Oozie provides or is Oozie more powerful than a YARN "application"...or simply more different from an application than I have conceptualized to the point that they don't necessarily step on one another's toes?
> Keith Wiley [EMAIL PROTECTED] keithwiley.com music.keithwiley.com
> "The easy confidence with which I know another man's religion is folly teaches
> me to suspect that my own is also."
> -- Mark Twain
Arun C. Murthy