Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Review request for HBASE-7692: Ordered byte[] serialization


Copy link to this message
-
Re: Review request for HBASE-7692: Ordered byte[] serialization
Jonathan Hsieh 2013-02-22, 14:31
Nick,

I'm +1 for it having its own module, and being a sibling of hbase-client.
 I'm assuming the client stuff will happen before we release 0.96 since it
has been started.

Jon.

On Fri, Feb 22, 2013 at 6:13 AM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:

> You're absolutely correct: this library introduces client-side conventions
> and is not needed from within the HMaster or RegionServer. Is
> the consensus that it should reside in it's own module or be a sibling to
> the o.a.h.hbase.client source tree? I'm a little confused by the current
> state of the modules; hbase-client looks empty while o.a.h.hbase.client
> sits under hbase-server.
>
> Thanks,
> Nick
>
> On Thu, Feb 21, 2013 at 11:56 PM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>
> > So I buy the argument about this being included in hbase, but several of
> > the questions still stand --
> >
> > Why is this part of hbase-common?  shouldn't this be just a dependency of
> > hbase-client module?  Does the hbase-server side need to depend on this?
> >
> > Since this is a large import of a currently isolated library, why not
> make
> > it a separate module instead of part of hbase-common?  This would
> enforce a
> > boundary that will prevent pollution from circular dependencies.
> >
> > Jon.
> >
> > On Thu, Feb 21, 2013 at 7:23 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:
> >
> > > I think this belongs in core HBase, as a replacement to Bytes, which
> > should
> > > be deprecated eventually. We have a Bytes utility which is supposed to
> > > convert basic java types to byte[]'s, but it does not work for signed
> > > numbers.
> > >
> > > We already know that all of the clients, Hive, Pig, Phoenix, have to
> have
> > > at least java type -> byte[] conversion utilities, and I think it is
> > > HBase's job to supply one so that different clients can interoperate.
> > Since
> > > internally we are also relying on serializing java types, we need that
> > > library in the core.
> > >
> > > BTW, I also think that we need to have a SQL-type to java type to
> byte[]
> > > layer, but that is another discussion.
> > >
> > > Enis
> > >
> > >
> > > On Thu, Feb 21, 2013 at 3:04 PM, Jonathan Hsieh <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Nick,
> > > >
> > > > While I believe having an order-preserving canonical serialization
> is a
> > > > good idea,  from doing a read of the mail and a skim of the jira it
> is
> > > not
> > > > clear to my why this is inside hbase as part of hbase-common.
> > > >
> > > > Why isn't this part of a library on top of hbase (a dependency for
> > > > Pig/Hive) instead of "inside" hbase?
> > > > Can't this functionality be done just from the client level?
> > > > What's the end goal hee? Is the goal here to replace the
> > Bytes.toBytes(*)
> > > > methods to enforced the ordering?
> > > > If I HBase has two mutually incompatible encodings "built-in", how
> > does a
> > > > dev know to use one or the other later on?
> > > > If this is essentially a mega import of a library (300k.. yikes) ,
> why
> > > not
> > > > make it a separate module instead of part of common?
> > > >
> > > > Jon.
> > > >
> > > > On Thu, Feb 21, 2013 at 10:35 AM, Nick Dimiduk <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > I'm of the opinion that HBase should provide a mechanism for
> > > serializing
> > > > > common java types such that the serialized format sorts according
> the
> > > > > the natural ordering of the type. I think many application efforts
> > end
> > > up
> > > > > building a custom, partial implementation of this kind of
> > functionality
> > > > on
> > > > > their own. I think HBase should provide a canonical implementation
> of
> > > > such
> > > > > a serialization format so that third-parties can reliably build on
> > top
> > > of
> > > > > HBase. Not just user applications, but other tools like Pig and
> Hive
> > > are
> > > > > also enabled. Implementations for
> > > > > HIVE-3634<https://issues.apache.org/jira/browse/HIVE-3634
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]