-RE: Debugging Mapreduce programs
Subroto Sanyal 2011-09-15, 05:18
MapReduce framework provide different in built approaches to debug a Job:
1) LocalJobRunner (Running the Job without a cluster; check basic
functionality of the Job implementation)
2) IsolationRunner (Running a failed TaskAttempt on cluster- Remote
debugging can be done)
Apart from this we can use Sysouts and loggings in the Map or Reduce Tasks.
These logs will come in "sysout","syslog","syserr" accordingly. These files
will be present in the userlogs folder.
User can pass Remote Debug argument to Child Process to enable remote
debugging of Child process.
If you have access through putty, then you can use remote Debug option and
connect through eclipse as well.
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
Sent: Thursday, September 15, 2011 9:49 AM
To: [EMAIL PROTECTED]
Subject: Re: Debugging Mapreduce programs
There are multiple ways to debug map reduce programs. I found comfortable
using the jobtracker web UI. You can your job tracker web UI at
Open it in a browser.
There you can find the running,failed and retired jobs. You can drill down
on any job to task tracker log levels from there.
Mostly I get the root cause of my job failures from there.
Hope it helps
Bejoy K S
From: <[EMAIL PROTECTED]>
Date: Thu, 15 Sep 2011 09:39:23 +0530
To: <[EMAIL PROTECTED]>
ReplyTo: [EMAIL PROTECTED]
Subject: Debugging Mapreduce programs
Can we use sysouts in java to debug MR programs.
Where would those sysouts be printed?
What would be the best way to debug a remote system where I have access to
the machine only through putty.
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged
If you are not the intended recipient, please contact the sender by reply
e-mail and destroy all copies of the original message.
Any unauthorised review, use, disclosure, dissemination, forwarding,
printing or copying of this email or any action taken in reliance on this
e-mail is strictly
prohibited and may be unlawful.