Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [UPDATE] Finishing up 0.96 --> WAS Re: 0.95 and 0.96 remaining issues


Copy link to this message
-
Re: [UPDATE] Finishing up 0.96 --> WAS Re: 0.95 and 0.96 remaining issues
On Tue, Jul 30, 2013 at 9:38 PM, James Taylor <[EMAIL PROTECTED]>wrote:

> So row key order won't match the natural sort order?
>

With the "Legacy" types that are based on Bytes, you get whatever you get.
Strings and pass-through byte[]'s work like normal; positive integers will
work, but not negatives; &c. HBase would not ship out-of-the-box with
general purpose order-preserving types, but you'd have the API and be able
to implement your own.
On Tue, Jul 30, 2013 at 9:11 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
> > On Tue, Jul 30, 2013 at 8:20 PM, James Taylor <[EMAIL PROTECTED]
> > >wrote:
> >
> > > What's the functionality that we'll lose without the order-preserving
> > part
> > > being included?
> > >
> >
> > Well, order preservation ;) Lacking 8201, we'd get all the existing Bytes
> > goodness but wrapped up in 8693's API. The framework is laid for other
> > HBase components, user applications, and downstream projects to start
> > building on it immediately. I'd like to start pluming it into some new
> > Filters, a couple of the MapReduce tools, Hive's interop layer, maybe
> even
> > replace PDataType with DataType if you're open to such a patch. The
> > OrderedBytes stuff will require applications to have a plan for data
> > migration when they decide to transition over to it, but they can get a
> > head-start on the boiler-plate code.
> >
> > On Tue, Jul 30, 2013 at 5:39 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > On Tue, Jul 30, 2013 at 3:28 PM, Enis Söztutar <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > Let me elaborate. There are at least new RPC PB, PB structures in
> > > HFiles
> > > > /
> > > > > hlogs, and zk, table locks, bucket cache, online merge, stochastic
> > LB,
> > > > > hbase on windows, *new data types*, AM changes, favorite node
> > > assignment,
> > > > > dist log replay, and tons of MTTR changes that are not run in
> > > production
> > > > so
> > > > > far.
> > > >
> > > >
> > > > Re: new data types, allow me to throw my hat into the ring as well. I
> > > spoke
> > > > with Stack this morning about a plan that I hope will allow the new
> > data
> > > > type API to squeeze in without requiring the order-preserving
> encoding.
> > > I'd
> > > > like to get this API out with 0.96 so that users interested in
> adopting
> > > > this feature can start migrating their applications sooner than
> later.
> > > > Jealously, I want to start work on the Hive/HBase plumbing using this
> > API
> > > > and the Hive guys are only willing to accept code that is built
> > against a
> > > > labeled HBase release. I'm hoping other projects (*cough* Phoenix,
> > > Impala,
> > > > Kiji *cough*) are keen to follow suit. It also allows for the feature
> > to
> > > > fan out internally -- Filters, Coprocessors, ImportTsv, &c. Using
> this
> > > > approach, DataType and Legacy* friends come in for 0.96.0 and
> > > OrderedBytes
> > > > can join the party in a 0.96.x when it's ready for adoption.
> > > >
> > > > Thanks,
> > > > Nick
> > > >
> > >
> >
>