Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> debug mapreduce basics


Copy link to this message
-
Re: debug mapreduce basics
Mike,

Check out this wiki
http://code.google.com/p/hadoop-clusternet/wiki/DebuggingJobsUsingEclipse

It shows how if your running in stand alone mode you can run a job in
debug mode so that you can then start a remote debugging session with
Eclipse. You can then step through your code.

I've found it to be quite helpful.

J
On Thu, 2011-04-28 at 14:59 -0400, Mike Spreitzer wrote:
> So I wrote my first org.apache.hadoop.madpreduce.Job
> (not ...mapred.Job).  Oddly enough, when the reducer is invoked, the
> "Iterable values" parameter actually iterates over just one value, not
> all the corresponding values that were output by my map tasks.  I know
> this from adding calls to System.out.println to my map and reduce
> code.  I also know that my combiner was not invoked (also from println
> debugging).  So now I am trying to determine why things are not being
> wired up correctly.  What is the right way to debug situations like
> this?
>
> Thanks,
> Mike Spreitzer
> SMTP: [EMAIL PROTECTED], Lotus Notes: Mike Spreitzer/Watson/IBM
> Office phone: +1-914-784-6424 (IBM T/L 863-)
> AOL Instant Messaging: M1k3Sprtzr
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB