-Re: [maybe off-topic?] article: Solving Big Data Challenges for Enterprise Application Performance Management
Andrew Purtell 2012-08-31, 13:48
Asynchbase redone with PB and attention to security would be a good place
to start. I can't commit resources in the immediate term, so that's easy
for me to say I know. Anyway seems we're on the same page wrt client.
On Friday, August 31, 2012, lars hofhansl wrote:
> Many of us have been saying for a while that the client needs love (i.e.
> needs to be rewritten) and that a new client should follow an async API
> (maybe with a thin synchronous veneer of top of it).
> The client is a big piece of HBase. And implementing all the aspects
> including security is a major task and nobody has committed the necessary
> resources for it, yet.
> asynchbase is a start, but it does not support many of the HBase features
> (coprocessors, security, etc).
> -- Lars
> '[EMAIL PROTECTED]');>>
> *Sent:* Thursday, August 30, 2012 2:41 PM
> *Subject:* Re: [maybe off-topic?] article: Solving Big Data Challenges
> for Enterprise Application Performance Management
> I do want to take a closer look at it. Not with the intent to replace the
> PB RPC with it but its odd to have two RPC stacks. What refactoring and
> code simplification/removal opportunities are here? Don't know (yet). More
> generally, to experiment with simple native async clients.
> On Thursday, August 30, 2012, lars hofhansl wrote:
> 0.94+ has the option to run a thrift-server-thread inside the
> RegionServers. Maybe we should improve upon that?
> From: Andrew Purtell <[EMAIL PROTECTED]>
> To: Andrew Purtell <[EMAIL PROTECTED]>
> Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Thursday, August 30, 2012 9:41 AM
> Subject: Re: [maybe off-topic?] article: Solving Big Data Challenges for
> Enterprise Application Performance Management
> Just want to clarify I mean experimenting with the approach of the Thrift
> client work not use of Thrift particularly.
> On Thursday, August 30, 2012, Andrew Purtell wrote:
> > This paper could very well have benchmarked the relative performance of
> > the YCSB drivers. Some take aways for me here are:
> > - Cluster setup is too difficult still
> > - There are opportunities for autotuning that would make it easier
> > users to get it right the first time and for academics and casual
> > benchmarkers alike to get a good result without becoming experts with
> > configuration
> > - The client library has been evolving toward fully async dispatch,
> > should focus on this, perhaps even consider reimplementing sync client
> on a
> > refactored async core. And look at making the Thrift based stuff FB put
> > front and center, because then native clients are possible.
> > - Given the above client work, the YCSB HBase driver should have a
> > rewrite.
> 'cvml', '[EMAIL PROTECTED]');>
> > > wrote:
> >> My reading of the paper is that they are actually not clear about
> >> or not HMasters were deployed on datanodes.
> >> I'm going to guess that they just used default configurations for HBase
> >> and
> >> YCSB, but the paper again is not specific enough.
> >> Why were they using 0.90.4 in 2012? Would have been nice to see some of
> >> the more recent work done in the area of performance.
> >> One thing the paper does touch on is the relative difficulty of standing
> >> up
> >> the cluster, which has not changed since 0.90.4. I think that's
> >> definitely
> >> something that could be improved upon.
> >> - Dave
> >> On Thu, Aug 30, 2012 at 6:27 AM, Cristofer Weber <
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)