Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # dev - Request for mentor for project on ACCUMULO-1197


+
Ajay Bhat 2013-07-11, 00:41
+
Keith Turner 2013-07-11, 13:57
+
Ajay Bhat 2013-07-11, 15:47
+
Keith Turner 2013-07-13, 00:52
+
Jonathan Hsieh 2013-07-15, 23:18
+
German Gutierrez 2013-07-16, 03:16
+
Sreejith Ramakrishnan 2013-07-16, 18:34
+
Sreejith Ramakrishnan 2013-07-17, 11:32
+
Keith Turner 2013-07-17, 12:31
+
Sreejith Ramakrishnan 2013-07-17, 18:01
+
Sreejith Ramakrishnan 2013-07-17, 18:08
Copy link to this message
-
Re: Request for mentor for project on ACCUMULO-1197
Billie Rinaldi 2013-07-17, 18:15
I realize there isn't a lot of time before the deadline, but if you are
interested in submitting a patch to HDFS I would encourage you to engage
with the HDFS community to see if they are interested in such a feature to
increase the likelihood of your patch being accepted.  Their dev mailing
list is [EMAIL PROTECTED].

I found one reference to Dapper in the HDFS tickets, HDFS-4680.  The ticket
isn't about tracing, but Todd Lipcon comments that he's experimented with
adding tracing to Hadoop RPC and asks if people would find that useful.
(There doesn't appear to be a reply.)

Billie
On Wed, Jul 17, 2013 at 11:01 AM, Sreejith Ramakrishnan <
[EMAIL PROTECTED]> wrote:

> I think its a good idea to modify HDFS and Accumulo to support HTrace since
> HBase has also started working in this direction. The progress made could
> benefit both projects. And also, HTrace follows the Dapper conventions.
>
> I'm not an expert. But from what I read in Google's paper and what's being
> provided in HTrace, we can modify HDFS instrumentation to add a 64-bit
> span-id to the RPC when tracing is enabled. And on the receiver side, if it
> receives a span-id, it shall also be traced with both the parent and child
> nodes having the same trace-id.
>
> What do you think? Please do correct me if I'm wrong.
>
> P.S: Also, the last date for application for the mentoring ends on 19th
> July. So, I request all the experts here to please let me know if you're
> ready to be a mentor. And I assure you, I won't bug you too much in your
> busy scheduler. It's a mandatory rule for the ICFOSS programme -
> http://community.apache.org/mentoringprogramme-icfoss-pilot.html
>
>
> On Wed, Jul 17, 2013 at 6:01 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>
> > On Wed, Jul 17, 2013 at 7:32 AM, Sreejith Ramakrishnan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Wouldn't modifying HDFS in any way mean forking the project? Wouldn't
> > that
> > > have problems in the long run?
> > >
> > >
> > Forking HDFS is not the way to go.  Getting your ideas and patches
> accepted
> > by HDFS is the way to go.
> >
> >
> >
> > >
> > > On Wed, Jul 17, 2013 at 12:04 AM, Sreejith Ramakrishnan <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > Hey,
> > > >
> > > > I have an idea to tackle this Jira. I've read the dapper paper and
> > > > currently examining the work done in thrift. Shall I make a detailed
> > post
> > > > here?
> > > >
> > > > I'm part of the ICFOSS Joint Mentoring Programme. So if someone can
> > > please
> > > > be my mentor, I'll be happy to work with you :)
> > > >
> > > > [1] http://community.apache.org/mentoringprogramme-icfoss-pilot.html
> > > >
> > > > Thank you,
> > > > Sreejith R
> > > >
> > > >
> > > > On Tue, Jul 16, 2013 at 8:46 AM, German Gutierrez <
> > > [EMAIL PROTECTED]
> > > > > wrote:
> > > >
> > > >> On Jul 12, 2013 8:53 PM, "Keith Turner" <[EMAIL PROTECTED]> wrote:
> > > >> >
> > > >> > On Thu, Jul 11, 2013 at 11:47 AM, Ajay Bhat <
> [EMAIL PROTECTED]>
> > > >> wrote:
> > > >> >
> > > >> > > Thanks Keith. I am looking into it now.
> > > >> > >
> > > >> > > Accumulo has done Dapper design based tracing and called it
> > > >> Cloudtrace.
> > > >> I
> > > >> > > am not aware of the ins and outs of this and would like to know
> > more
> > > >> > > indepth about it. Can anyone help out here?
> > > >> > >
> > > >> >
> > > >> > Cloudtrace is an implementation of the ideas mentioned in the
> Dapper
> > > >> paper.
> > > >> >  Its only used by Accumulo at this point.   The reason I mentioned
> > the
> > > >> > HBase work is so you could asses the status of that work.  You
> > should
> > > >> > determine if any other work besides the HBase and Accumulo efforts
> > > >> exists.
> > > >> >
> > > >> > I can think of a few ways to tackle this problem.
> > > >> >
> > > >> >  1. Modify HDFS to use cloudtrace.  Cloudtrace is currently
> > > implemented
> > > >> on
> > > >> > top of thrift, HDFS does not use thrift.
> > > >> >  2. Modify HDFS and Accumulo to use HBase  tracing.
+
Sreejith Ramakrishnan 2013-07-17, 18:23
+
Billie Rinaldi 2013-07-17, 18:33
+
Sreejith Ramakrishnan 2013-07-17, 18:40
+
Billie Rinaldi 2013-07-17, 18:51
+
Sreejith Ramakrishnan 2013-07-17, 19:07
+
Sreejith Ramakrishnan 2013-07-17, 21:31
+
Billie Rinaldi 2013-07-18, 01:51
+
Sreejith Ramakrishnan 2013-07-18, 09:33
+
Keith Turner 2013-07-18, 11:26
+
Sreejith Ramakrishnan 2013-07-18, 13:45