Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - [UPDATE] Finishing up 0.96 --> WAS Re: 0.95 and 0.96 remaining issues


Copy link to this message
-
Re: [UPDATE] Finishing up 0.96 --> WAS Re: 0.95 and 0.96 remaining issues
James Taylor 2013-07-31, 04:38
So row key order won't match the natural sort order?
On Tue, Jul 30, 2013 at 9:11 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:

> On Tue, Jul 30, 2013 at 8:20 PM, James Taylor <[EMAIL PROTECTED]
> >wrote:
>
> > What's the functionality that we'll lose without the order-preserving
> part
> > being included?
> >
>
> Well, order preservation ;) Lacking 8201, we'd get all the existing Bytes
> goodness but wrapped up in 8693's API. The framework is laid for other
> HBase components, user applications, and downstream projects to start
> building on it immediately. I'd like to start pluming it into some new
> Filters, a couple of the MapReduce tools, Hive's interop layer, maybe even
> replace PDataType with DataType if you're open to such a patch. The
> OrderedBytes stuff will require applications to have a plan for data
> migration when they decide to transition over to it, but they can get a
> head-start on the boiler-plate code.
>
> On Tue, Jul 30, 2013 at 5:39 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
> >
> > > On Tue, Jul 30, 2013 at 3:28 PM, Enis Söztutar <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Let me elaborate. There are at least new RPC PB, PB structures in
> > HFiles
> > > /
> > > > hlogs, and zk, table locks, bucket cache, online merge, stochastic
> LB,
> > > > hbase on windows, *new data types*, AM changes, favorite node
> > assignment,
> > > > dist log replay, and tons of MTTR changes that are not run in
> > production
> > > so
> > > > far.
> > >
> > >
> > > Re: new data types, allow me to throw my hat into the ring as well. I
> > spoke
> > > with Stack this morning about a plan that I hope will allow the new
> data
> > > type API to squeeze in without requiring the order-preserving encoding.
> > I'd
> > > like to get this API out with 0.96 so that users interested in adopting
> > > this feature can start migrating their applications sooner than later.
> > > Jealously, I want to start work on the Hive/HBase plumbing using this
> API
> > > and the Hive guys are only willing to accept code that is built
> against a
> > > labeled HBase release. I'm hoping other projects (*cough* Phoenix,
> > Impala,
> > > Kiji *cough*) are keen to follow suit. It also allows for the feature
> to
> > > fan out internally -- Filters, Coprocessors, ImportTsv, &c. Using this
> > > approach, DataType and Legacy* friends come in for 0.96.0 and
> > OrderedBytes
> > > can join the party in a 0.96.x when it's ready for adoption.
> > >
> > > Thanks,
> > > Nick
> > >
> >
>