Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Visual debugging tools for hadoop


Copy link to this message
-
RE: Visual debugging tools for hadoop
I'll start with Ambari and would like to extend it to give it the ability to inject in breakpoints dynamically as well as stopping a job in the middle and hot swap in code.

> From: [EMAIL PROTECTED]
> To: [EMAIL PROTECTED]
> Subject: Re: Visual debugging tools for hadoop
> Date: Tue, 18 Jun 2013 17:53:47 +0000
>
> Yes data flow visualizations definitely sound like something that would be
> good for Ambari.  If you are interested in debugging Hadoop jobs there is
> also the Hadoop Development Tools project
>
> http://incubator.apache.org/projects/hdt.html
>
> It is taking the Eclipse plugin for Hadoop and really improving it.  I
> know that there has been some work to try and get a debugger working over
> there where you could walk through parts of your MR job line by line.
>
> --Bobby
>
> On 6/14/13 12:40 PM, "Chris Nauroth" <[EMAIL PROTECTED]> wrote:
>
> >Hi Saikat,
> >
> >You might want to investigate contributing on Apache Ambari, which has
> >features for visualization of jobs and end-to-end flows consisting of
> >multiple dependent jobs.
> >
> >http://incubator.apache.org/ambari/
> >
> >Chris Nauroth
> >Hortonworks
> >http://hortonworks.com/
> >
> >
> >
> >On Fri, Jun 14, 2013 at 8:20 AM, Saikat Kanjilal
> ><[EMAIL PROTECTED]>wrote:
> >
> >> Hi Folks,
> >> I was wondering if anyone is currently working on or thinking about
> >>visual
> >> debugging tools for mapreduce jobs, I was thinking about starting an
> >>effort
> >> to build an end to end visual tool that shows all the steps in the
> >> mapreduce workflow and data flows, variable content changing to speed up
> >> debugging of jobs.    Please ignore if something like this already
> >>exists
> >> and if not I'd love to collaborate with folks to build something.
> >>
> >>
> >> Regards
> >>
>
     
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB