-RE: Hadoop JARs and Eclipse
John Lilley 2013-06-05, 19:51
Thanks for your response. I don't need the whole of Hadoop projects in Eclipse, I just need to be able to build Hadoop and step into its source files. Please help me better understand what you recommend as I have a very cursory understanding of maven and its interaction with Eclipse.
-- From where will it fetch the Hadoop JARs?
-- If I have built Hadoop from source, how do I tell it to find my just-built JARs instead of those from the official release site?
-- Is it necessary to build Hadoop from source so that the Eclipse debugger can locate source files and display them if I need to step in?
My goals are to build an application on Hadoop 2.0. Currently I am using the 2.0.4 alpha release. Eventually I this application will include an ApplicationMaster, auxiliary service, and a pile of embedded code and data. However, I don't believe that I will be able to use Hadoop releases as-is without trouble. I suspect that the following will be necessary at some point:
-- Step into the source code of referenced Hadoop code in the debugger.
-- Make changes to the Hadoop source and rebuild it if things are not working as expected (e.g. to add logging).
-- Switch to building from SVN instead of a release snapshot, and apply patches.
From: Harsh J [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, June 05, 2013 11:43 AM
To: <[EMAIL PROTECTED]>
Subject: Re: Hadoop JARs and Eclipse
If your goal is to simply build an application, then you can use a Maven project. Why do you require the whole of Hadoop's projects itself on Eclipse when you can simply have the dependencies via a maven pom.xml?
The following is what you can use in a simple maven app, to include all necessary dependencies automatically, for example:
Then importing this project into Eclipse (using m2e plugin) would be equivalent to "Adding all relevant Hadoop JARs to Eclipse".
On Wed, Jun 5, 2013 at 11:07 PM, John Lilley <[EMAIL PROTECTED]> wrote:
> Well, I've failed and given up on building Hadoop in Eclipse. Too
> many things go wrong with Maven plugins and m2e.
> But Hadoop builds just fine using the command-line, and it runs using
> Sandy's development-node instructions.
> My strategy now is
> 1) Tell Eclipse about all of the Hadoop JARs
> 2) Make a project with the sample ApplicationMaster distributed shell
> and build it.
> Is there a shortcut for adding all relevant JARs to Eclipse? They
> ended up in a lot of separate folders after the build and I can't tell
> if there is one "target" containing all of the ones I need.