Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Debug MR tasks impossible.


+
Pedro Costa 2012-03-29, 15:33
+
Brock Noland 2012-03-29, 16:07
+
Wydra, Paul 2012-03-29, 16:13
+
Steve Lewis 2012-03-29, 16:32
+
Ravi Prakash 2012-03-29, 16:06
+
Devaraj k 2012-03-29, 16:11
Copy link to this message
-
Re: Debug MR tasks impossible.
On 0.23 I usually breakpoint on DefaultContainerExecutor.java : 139. Modify
the script mentioned in nmPrivateContainerScriptPath to include JAVA remote
debugging options. Connect to the nodemanager and then breakpoint again in
MRAppMaster.java. Tedious. I know. :(

On Thu, Mar 29, 2012 at 11:06 AM, Ravi Prakash <[EMAIL PROTECTED]> wrote:

> Hi Pedro,
>
> I know its sub-optimal but you should be able to put in as many
> System.out.println / log messages as you want and you should be able to see
> them in stdout, and syslog files. Which version of hadoop are you using?
>
>
>
>
> On Thu, Mar 29, 2012 at 10:33 AM, Pedro Costa <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> I'm trying to debug map and reduce tasks for a quite long time, and it
>> seems that it's impossible. MR are launched in new process and there's no
>> way to debug them. Even with IsolationRunner class it's impossible. This
>> isn't good because I really need to debug the class, to understand some
>> changes that I made to the code.
>>
>> I wonder how MapReduce programmers could debug the code that they
>> implemented?
>>
>> --
>> Best regards,
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB