Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> DISCUSS : HFile V3 proposal for tags in 0.96


Copy link to this message
-
Re: DISCUSS : HFile V3 proposal for tags in 0.96
[ Reposting from a different account - looks like infra borked LDAP
somehow. ]

On Fri, Jul 19, 2013 at 4:31 PM, Stack <[EMAIL PROTECTED]> wrote:

> 0.96 is way too late already.  I am shooting for end of July as cut-off
> point when I intend rolling a 0.95.2 RC.
>

We are shooting for a HFileV3 in time for inclusion in this RC.

> What I would suggest you fellows do is given you have enough votes
> and reviewers between you, then I would drive hard at getting the
> needed core and any api breaking changes committed over the next
> week or two so you have these in before the gate comes down.

We were planning to do it this way but now until it became clear that 0.96
WILL GO OUT at the end of this month. :-) We do not want to change APIs or
make invasive changes in 0.95/0.96 now because it is so close to going out.

We would be unhappy with a release of 0.96 if we would be unable to make
progress with security use cases in a shipping 0.96. I'm not sure how long
we would have to wait for 0.98. At the same time we don't want to make a
user pay a price for security if they don't want it, that is what has been
occupying us on this work for the past couple of months. We don't want to
hold up the 0.96 release either and see HFileV3 as a way everyone wins.

So we refactored the most recent prototype of tag persistence in file into
HFileV3 in a way that minimizes changes to core code. We're only changing
the data block encoder base class, because the design intent of that
package is to be shared among all file formats. No changes to anything on
the client or RPC. These changes are being looked at by three committers
right now.

We abandoned further work on Cell and CellCodec etc. until 0.98, agree, the
clock has run out there. Plumbing tags through to the client will be part
of that. For the use cases we have on deck for tags, only server side
support is needed and clients can set operation attributes to get them to
the server for now.

On Fri, Jul 19, 2013 at 4:31 PM, Stack <[EMAIL PROTECTED]> wrote:

> On Fri, Jul 19, 2013 at 3:34 PM, Andrew Purtell <[EMAIL PROTECTED]>
> wrote:
>
> > On Fri, Jul 19, 2013 at 2:02 PM, Elliott Clark <[EMAIL PROTECTED]>
> wrote:
> >
> > > That should mean that it's possible to make this change in a later
> > version
> > > without holding up 0.96.
> > >
> >
> > Will this hold up 0.96? What would you suggest we do, besides making a
> > patch available for review along with test results, which is in the
> works.
> > As you say this work can co-exist perfectly well with all code in 0.96
> > without interference with it. (Although it is reasonable to be skeptical
> of
> > my claims until viewing a patch.)
> >
> >
> 0.96 is way too late already.  I am shooting for end of July as cut-off
> point when I intend rolling a 0.95.2 RC.  I do not want to take on new
> features after 0.95.2.
>
> What I would suggest you fellows do is given you have enough votes and
> reviewers between you, then I would drive hard at getting the needed core
> and any api breaking changes committed over the next week or two so you
> have these in before the gate comes down.  Perhaps the new file format will
> make it in in time but I do not think we should hold up 0.96 till it is
> baked; it can come in post 0.96 especially if it is all additions and not
> core amendments.
>
> Is there a design we can be checking out meantime?
>
> St.Ack
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB