Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Debug Hadoop Junit Test in Eclipse


Copy link to this message
-
Re: Debug Hadoop Junit Test in Eclipse
In that case i guess you will have to statically trace the code your self.

On Mon, Dec 16, 2013 at 10:32 AM, Karim Awara <[EMAIL PROTECTED]>wrote:

>
> Useful post, however, I am not trying to  debug mapreduce programs with
> its associated VMs. I want to modify HDFS source code on how it uploads
> files. So I am only looking to trace fs commands through the DFS shell. I
> believe this should be require less work in debugging than actually going
> to mapred VMs!
>
>
>
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>
>> Excellent question.  its not trivial to debug a distributed app in
>> eclipse, but it is totally doable using javaagents .   We've written it up
>> here:
>>
>>
>> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>>
>> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
>> with the tutorial, he might have some extra advice also (cc'd on this
>> email), I've written up one way to do this using the bigtop VMs here.
>>
>>
>>
>>
>> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>>> test timed out exception. I believe this is because I am trying to stop one
>>> thread while the rest are working. I have changed the breakpoint property
>>> to suspend VM, but still same problem.
>>>
>>>
>>> How can I trace calls made by datanode/namenode when running
>>> TestDFSShell.java Junit test through eclipse?
>>>
>>> I am using hadoop 2.2.0
>>>
>>> --
>>> Best Regards,
>>> Karim Ahmed Awara
>>>
>>> ------------------------------
>>> This message and its contents, including attachments are intended solely
>>> for the original recipient. If you are not the intended recipient or have
>>> received this message in error, please notify me immediately and delete
>>> this message from your computer system. Any unauthorized use or
>>> distribution is prohibited. Please consider the environment before printing
>>> this email.
>>
>>
>>
>>
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>

--
Jay Vyas
http://jayunit100.blogspot.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB