Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Visual debugging tools for hadoop


Copy link to this message
-
RE: Visual debugging tools for hadoop
I'll start with Ambari and would like to extend it to give it the ability to inject in breakpoints dynamically as well as stopping a job in the middle and hot swap in code.

> From: [EMAIL PROTECTED]
> To: [EMAIL PROTECTED]
> Subject: Re: Visual debugging tools for hadoop
> Date: Tue, 18 Jun 2013 17:53:47 +0000
>
> Yes data flow visualizations definitely sound like something that would be
> good for Ambari.  If you are interested in debugging Hadoop jobs there is
> also the Hadoop Development Tools project
>
> http://incubator.apache.org/projects/hdt.html
>
> It is taking the Eclipse plugin for Hadoop and really improving it.  I
> know that there has been some work to try and get a debugger working over
> there where you could walk through parts of your MR job line by line.
>
> --Bobby
>
> On 6/14/13 12:40 PM, "Chris Nauroth" <[EMAIL PROTECTED]> wrote:
>
> >Hi Saikat,
> >
> >You might want to investigate contributing on Apache Ambari, which has
> >features for visualization of jobs and end-to-end flows consisting of
> >multiple dependent jobs.
> >
> >http://incubator.apache.org/ambari/
> >
> >Chris Nauroth
> >Hortonworks
> >http://hortonworks.com/
> >
> >
> >
> >On Fri, Jun 14, 2013 at 8:20 AM, Saikat Kanjilal
> ><[EMAIL PROTECTED]>wrote:
> >
> >> Hi Folks,
> >> I was wondering if anyone is currently working on or thinking about
> >>visual
> >> debugging tools for mapreduce jobs, I was thinking about starting an
> >>effort
> >> to build an end to end visual tool that shows all the steps in the
> >> mapreduce workflow and data flows, variable content changing to speed up
> >> debugging of jobs.    Please ignore if something like this already
> >>exists
> >> and if not I'd love to collaborate with folks to build something.
> >>
> >>
> >> Regards
> >>
>