Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Maven build YARN ResourceManager only

Chin-Jung Hsu 2013-04-13, 03:46
Copy link to this message
Re: Maven build YARN ResourceManager only
Chris Nauroth 2013-04-13, 04:41
I don't have an answer to your exact question, but I do have a different
suggestion that prevents the need to do frequent rebuilds of the whole
Hadoop source tree.  First, do a full build of the distribution tar.gz.
 Extract it and set up a custom hadoop-env.sh for yourself.  Inside the
hadoop-env.sh file, export the environment variables
you want to prepend before the classes loaded from the distribution.  For
example, this is what I have in mine right now, because I'm mostly working
on HDFS and NodeManager:


For your ResourceManager work, you could set up your HADOOP_CLASSPATH to
point at your hadoop-yarn-server-resourcemanager/target/classes directory.
 Then, source (.) this hadoop-env.sh in any shell that you're using to run
hadoop commands.  The daemons will print their full classpath before
launching, so you can check that to see if it worked.

With all of this in place, you can keep recompiling just
hadoop-yarn-server-resourcemanager whenever you make changes instead of the
whole hadoop-common tree.  Does this help?

On Fri, Apr 12, 2013 at 8:46 PM, Chin-Jung Hsu <[EMAIL PROTECTED]>wrote:

> I am implementing my own YARN scheduler under 2.0.3-alpha.  Is that
> possible to build only the ResourceManager project, and then create a
> distribution tar.gz for the entire Hadoop project?  Right now, the
> compiling time takes me about 9 minutes.
> Thanks,
> oxhead
Chin-Jung Hsu 2013-04-13, 14:10