Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Oozie vs. YARN "application"


Copy link to this message
-
Re: Oozie vs. YARN "application"
Just to be clear yarn does offer something described as dag management, but perhaps you mean it doesn't offer a dag *of* applications, only of mr jobs *within* an application. Is that what you mean?
________________________________________
Sent from my phone, please excuse my brevity.
Keith Wiley, [EMAIL PROTECTED], http://keithwiley.com
Arun C Murthy <[EMAIL PROTECTED]> wrote:

YARN doesn't yet have a DAGApplicationMaster which can handle a DAG of jobs.

Conceivable Oozie could offload the DAG management once it's available.

OTOH, Oozie provides much more than just DAG management - it provides time & data-availability based scheduling of workflows.

hth,
Arun

On May 17, 2012, at 4:47 PM, Keith Wiley wrote:

> Hadoop 0.23 (MRv2 or YARN) provides the concept of an "application", which is described as either a classic single MR job *or* a DAG of such jobs...which at a glance appears redundant with Oozie's primary purpose.
>
> How do they differ? Does 0.23 effectively obviate the service that Oozie provides or is Oozie more powerful than a YARN "application"...or simply more different from an application than I have conceptualized to the point that they don't necessarily step on one another's toes?
>
>_____________________________________________

> Keith Wiley [EMAIL PROTECTED] keithwiley.com music.keithwiley.com
>
> "The easy confidence with which I know another man's religion is folly teaches
> me to suspect that my own is also."
> -- Mark Twain
>_____________________________________________

>

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB