Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Oozie vs. YARN "application"

Copy link to this message
Re: Oozie vs. YARN "application"
Just to be clear yarn does offer something described as dag management, but perhaps you mean it doesn't offer a dag *of* applications, only of mr jobs *within* an application. Is that what you mean?
Sent from my phone, please excuse my brevity.
Keith Wiley, [EMAIL PROTECTED], http://keithwiley.com
Arun C Murthy <[EMAIL PROTECTED]> wrote:

YARN doesn't yet have a DAGApplicationMaster which can handle a DAG of jobs.

Conceivable Oozie could offload the DAG management once it's available.

OTOH, Oozie provides much more than just DAG management - it provides time & data-availability based scheduling of workflows.


On May 17, 2012, at 4:47 PM, Keith Wiley wrote:

> Hadoop 0.23 (MRv2 or YARN) provides the concept of an "application", which is described as either a classic single MR job *or* a DAG of such jobs...which at a glance appears redundant with Oozie's primary purpose.
> How do they differ? Does 0.23 effectively obviate the service that Oozie provides or is Oozie more powerful than a YARN "application"...or simply more different from an application than I have conceptualized to the point that they don't necessarily step on one another's toes?

> Keith Wiley [EMAIL PROTECTED] keithwiley.com music.keithwiley.com
> "The easy confidence with which I know another man's religion is folly teaches
> me to suspect that my own is also."
> -- Mark Twain


Arun C. Murthy
Hortonworks Inc.