Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: Debug Hadoop Junit Test in Eclipse


Copy link to this message
-
Re: Debug Hadoop Junit Test in Eclipse
Jay Vyas 2013-12-16, 16:07
In that case i guess you will have to statically trace the code your self.

On Mon, Dec 16, 2013 at 10:32 AM, Karim Awara <[EMAIL PROTECTED]>wrote:

>
> Useful post, however, I am not trying to  debug mapreduce programs with
> its associated VMs. I want to modify HDFS source code on how it uploads
> files. So I am only looking to trace fs commands through the DFS shell. I
> believe this should be require less work in debugging than actually going
> to mapred VMs!
>
>
>
>
> --
> Best Regards,
> Karim Ahmed Awara
>
>
> On Mon, Dec 16, 2013 at 5:57 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>
>> Excellent question.  its not trivial to debug a distributed app in
>> eclipse, but it is totally doable using javaagents .   We've written it up
>> here:
>>
>>
>> http://jayunit100.blogspot.com/2013/07/deep-dive-into-hadoop-with-bigtop-and.html
>>
>> FYI cc Brad childs (https://github.com/childsb) at red hat has helped me
>> with the tutorial, he might have some extra advice also (cc'd on this
>> email), I've written up one way to do this using the bigtop VMs here.
>>
>>
>>
>>
>> On Mon, Dec 16, 2013 at 8:07 AM, Karim Awara <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> I want to trace how a file upload (-put) happens in hadoop. So Im junit
>>> testing TestDFSShell.java. When I try to debug the test, It fails due to
>>> test timed out exception. I believe this is because I am trying to stop one
>>> thread while the rest are working. I have changed the breakpoint property
>>> to suspend VM, but still same problem.
>>>
>>>
>>> How can I trace calls made by datanode/namenode when running
>>> TestDFSShell.java Junit test through eclipse?
>>>
>>> I am using hadoop 2.2.0
>>>
>>> --
>>> Best Regards,
>>> Karim Ahmed Awara
>>>
>>> ------------------------------
>>> This message and its contents, including attachments are intended solely
>>> for the original recipient. If you are not the intended recipient or have
>>> received this message in error, please notify me immediately and delete
>>> this message from your computer system. Any unauthorized use or
>>> distribution is prohibited. Please consider the environment before printing
>>> this email.
>>
>>
>>
>>
>> --
>> Jay Vyas
>> http://jayunit100.blogspot.com
>>
>
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
>

--
Jay Vyas
http://jayunit100.blogspot.com