Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Maven build YARN ResourceManager only


Copy link to this message
-
Re: Maven build YARN ResourceManager only
I don't have an answer to your exact question, but I do have a different
suggestion that prevents the need to do frequent rebuilds of the whole
Hadoop source tree.  First, do a full build of the distribution tar.gz.
 Extract it and set up a custom hadoop-env.sh for yourself.  Inside the
hadoop-env.sh file, export the environment variables
HADOOP_USER_CLASSPATH_FIRST=true and HADOOP_CLASSPATH= any classpath that
you want to prepend before the classes loaded from the distribution.  For
example, this is what I have in mine right now, because I'm mostly working
on HDFS and NodeManager:

export HADOOP_USER_CLASSPATH_FIRST=true
HADOOP_REPO=~/git/hadoop-common
export
HADOOP_CLASSPATH=$HADOOP_REPO/hadoop-common-project/hadoop-common/target/classes:$HADOOP_REPO/hadoop-hdfs-project/hadoop-hdfs/target/classes:$HADOOP_REPO/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/classes:$HADOOP_REPO/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/classes

For your ResourceManager work, you could set up your HADOOP_CLASSPATH to
point at your hadoop-yarn-server-resourcemanager/target/classes directory.
 Then, source (.) this hadoop-env.sh in any shell that you're using to run
hadoop commands.  The daemons will print their full classpath before
launching, so you can check that to see if it worked.

With all of this in place, you can keep recompiling just
hadoop-yarn-server-resourcemanager whenever you make changes instead of the
whole hadoop-common tree.  Does this help?

Thanks,
--Chris
On Fri, Apr 12, 2013 at 8:46 PM, Chin-Jung Hsu <[EMAIL PROTECTED]>wrote:

> I am implementing my own YARN scheduler under 2.0.3-alpha.  Is that
> possible to build only the ResourceManager project, and then create a
> distribution tar.gz for the entire Hadoop project?  Right now, the
> compiling time takes me about 9 minutes.
>
> Thanks,
> oxhead
>