Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Review request for HBASE-7692: Ordered byte[] serialization


Copy link to this message
-
Re: Review request for HBASE-7692: Ordered byte[] serialization
Matt Corgan 2013-02-23, 00:48
I agree with Jonathan that ideally this would not depend on hbase or
hadoop.  Could we just replace Hadoop's BytesWritable with a new class that
does the same thing?

I also have a concern about the way it builds the multi-field byte[] by
allocating somewhat expensive Builder objects, etc.  It's suitable for
application level code, but most of the innards of hbase regionserver
should be using tighter code for best performance and less garbage.
 Perhaps in a future issue we can separate the builder wrappers from their
internal byte converters so that hbase-server can use the lower-level byte
converters without the builder overhead.
On Fri, Feb 22, 2013 at 4:33 PM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:

> I think I misspoke slightly but basically agree with Matt's notion that
> this would end up being the place to pickup the orderly jar and that
> ideally it has no hbase-* dependencies.
>
> I actually feel that the hbase-orderly module is a sibling to hbase-common
> and hbase-client. My initial thought is that this is ideally not depended
> upon by the hbase-client.  An app would use hbase-orderly and hbase-client.
>
>
>  A simplified module dependency graph (excluding some details) would be
> (where -> == "depends on")
>
> app -> hbase-client, hbase-orderly
> hbase-client -> hbase-protocol, hbase-common, *-compat
> hbase-common -> none of the hbase-*
> hbase-orderly -> none of the hbase-*
>
> I'm don't quite understand what the multiple patches are for the module
> work (or is this follow on stuff that uses this)?  can you explain what the
> breakdown would be?  since it isn't committed yet and should be self
> contained, just do the big import as a single patch?
>
> Thanks for bring this up for discussion Nick.
>
> Jon.
>
> On Fri, Feb 22, 2013 at 3:13 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
> > On Fri, Feb 22, 2013 at 10:00 AM, Matt Corgan <[EMAIL PROTECTED]>
> wrote:
> >
> > > To nitpick a little it wouldn't quite be a sibling of hbase-client
> > because
> > > hbase-client depends on hbase-common and hbase-protocol
> > >
> >
> > Actually, quite the contrary. I don't see this as being an external
> module
> > as much as integral to the client's use of HBase (read "client" as
> > "application consuming HBase", not "the HBase RPC client
> implementation").
> > Further, once HBase provides a suitable serialization format for
> > primitives, why not push them into the client API? IMHO, HBase really
> > should provide basic types for users at the Mutation layer. That,
> however,
> > belongs in an entirely separate ticket.
> >
> > On Fri, Feb 22, 2013 at 9:32 AM, Elliott Clark <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > Yep the client will be fully separated as soon as rpc changes
> > > > are stabilized.  Until then keeping up the move patch was just too
> > > onerous.
> > > >
> > > >
> > > > On Fri, Feb 22, 2013 at 6:31 AM, Jonathan Hsieh <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > Nick,
> > > > >
> > > > > I'm +1 for it having its own module, and being a sibling of
> > > hbase-client.
> > > > >  I'm assuming the client stuff will happen before we release 0.96
> > since
> > > > it
> > > > > has been started.
> > > > >
> > > > > Jon.
> > > > >
> > > > > On Fri, Feb 22, 2013 at 6:13 AM, Nick Dimiduk <[EMAIL PROTECTED]>
> > > > wrote:
> > > > >
> > > > > > You're absolutely correct: this library introduces client-side
> > > > > conventions
> > > > > > and is not needed from within the HMaster or RegionServer. Is
> > > > > > the consensus that it should reside in it's own module or be a
> > > sibling
> > > > to
> > > > > > the o.a.h.hbase.client source tree? I'm a little confused by the
> > > > current
> > > > > > state of the modules; hbase-client looks empty while
> > > o.a.h.hbase.client
> > > > > > sits under hbase-server.
> > > > > >
> > > > > > Thanks,
> > > > > > Nick
> > > > > >
> > > > > > On Thu, Feb 21, 2013 at 11:56 PM, Jonathan Hsieh <
> [EMAIL PROTECTED]
> > >
> > > > > wrote:
> > > > > >
> > > > > > > So I buy the argument about this being included in hbase, but