Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Hadoop in Eclipse


Copy link to this message
-
Re: Hadoop in Eclipse
Seconded. In Eclipse, I create a new empty Java project, then import as an external jar the core hadoop jar. That's it. It'll then offer code completion and online code checking as you type. To build my code I use ant on the command line and a build.xml file that I derived from the hadoop build.xml, but vastly simplified. I run the driver jar locally which then runs M/R either locally or on a cluster...or I can upload the jar to Azure...or presumably AWS although I haven't worked with that cloud yet.

The Hadoop core jar should get you going.

Cheers!
________________________________________
Sent from my phone, please excuse my brevity.
Keith Wiley, [EMAIL PROTECTED], http://keithwiley.com
Jagat <[EMAIL PROTECTED]> wrote:

Hello,

If you are looking to setup eclipse just to write mappers and reducers for
your code then

Create a eclipse java project
Download Apache bundle of Hadoop from Apache hadoop website
If you are using Hadoop 0.23 then go to share folder and add all those jars
e.g mapreduce , hdfs etc into project build path
If you are using Hadoop 1.x or 0,20 series then jars are present right
inside the home folder so you can directly add them to build path

Just run the main class which is your driver for your mapper and reducer
and remember to use job.setJarByClass method

Hope this helps.
On Tue, May 8, 2012 at 3:40 AM, Wilson Wayne - wwilso <
[EMAIL PROTECTED]> wrote:

> I’m new to Hadoop and am trying to get it setup in Eclipse. I��m
> following the “Working with Hadoop under Eclipse” wiki to do this. First
> let me make sure this will do what I am hoping it will do. I’m not so much
> interested in modifying the Hadoop code itself (although I may in the not
> too distant future). I’m more looking to be able to code and test mappers
> and reducers in Eclipse and leverage the debugging and other tools in
> Eclipse in that process. If that���s not what the wiki is describing than
> let me know that and ignore my issue. If there is a way to do this, would
> you please point me to where I can find the instructions. If this is what
> the wiki is describing than I have an issue I need help with.****
>
> ** **
>
> I’ve followed the directions on the Wiki and even worked through a couple
> of issues by searching the archive. However, now I’ve hit an error that
> while I did find it in the archive, the fix that was presented is not
> fixing my issue. After importing all the projects, I get a build error:
> “Project ‘MapReduceTools’ is missing required library:
> ���lib/hadoop-core.jar’”. I’ve added the M2_REPO variable as someone
> suggested in the archive where I found this error once. That hasn’t fixed
> or even changed the error. I’ve looked in my maven repository
> (c:\Users\wwilso\.m2\repository) and there is not a “Hadoop-core.jar”
> anywhere in the repository that windows explorer can find. In fact,
> windows explorer can’t find a “Hadoop-core.jar” anywhere on my c:\ drive.
> ****
>
> ** **
>
> I did my build using the directions from the wiki (got source using git in
> cygwin and then executing the two maven commands in cywgin). I did use the
> patch for the “scripts/saveVersion.sh” not found error to resolve that
> issue. That’s the only thing I have changed from the wiki or code. Using
> Maven-3.0.4, ant-1.8.3, and protobuf-2.4.1. Anyone have any ideas? Thanks
> in advance.****
>
> ** **
>
> ** **
>
> WAYNE WILSON
> Product Technical Consultant
> Acxiom Corporation
> EML [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> TEL 501.342.2870
> MBL 972.984.0268
> 5213 Rolling Rock Dr, McKinney, TX, 75070, USA
> WWW.ACXIOM.COM
>
> ****
>
> [image: Friend Us on Facebook] <http://www.facebook.com/acxiomcorp>; [image:
> Link Us on LinkedIn]<http://www.linkedin.com/groupRegistration?gid=2901735>;
> [image: Follow Us on Twitter] <http://twitter.com/acxiom>;****
>
> ** **
>
> [image: cid:[EMAIL PROTECTED]4350]****
>_____________________________________________

>
> The information contained in this communication is confidential, is
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB