Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: Can I use YARN with embedded job history server


Copy link to this message
-
Re: Can I use YARN with embedded job history server
Harsh J 2012-09-07, 09:39
Hi Henning,

On Fri, Sep 7, 2012 at 2:31 PM, Henning Blohm <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I am using CDH4.01 from a tarball and so far everything works kind of ok
> (albeit quite different from CDH3). It seems though that I fail to get an
> embedded job history server running when using YARN. On the other hand,
> there is no $HADOOP_HOME/sbin/start-historyserver.sh or the like. I can
> start the history server using mr-jobhistory-deamon.sh with appropriate
> params.

Please use HADOOP_PREFIX instead of HADOOP_HOME. The latter's been deprecated.

As you've found out, using the following is the correct way for
starting the MR2 Job History Server when starting via a tarball:

$ sbin/start-historyserver.sh start historyserver

> In other words: Is there a way to have an embedded job history server (small
> setup) when using YARN?

By "embedded" do you mean a non-separate daemon for MR jobs? Or do you
mean start-yarn.sh itself starting the

YARN and MR2 (which uses YARN) are two separate set of components, and
hence start-yarn.sh doesn't also start the MR-specific job history
server. The JHS doesn't apply/work for any other apps of YARN other
than MR2 right now, so its not unified.

> Thanks,
>   Henning
>
>

--
Harsh J