Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> DISCUSS : HFile V3 proposal for tags in 0.96


Copy link to this message
-
Re: DISCUSS : HFile V3 proposal for tags in 0.96
>>Based on your details, I think it will be, but very minimal, or
almost invisible, correct?
Yes of course.
Regarding migration, any file written with V2 would still be read with
HFileReaderV2 and the new files will be written with V3.  So there should
not be any problem here.  We are anyway testing these things to  make sure
we don't break anywhere.  Thanks Jean for the interest.

@Stack
I would write up on the changes foreseen for the Codec changes to support
RPC and HFileV3.
Discussing with Anoop, we have some benefits when the Tags are written as
the byte array and when tags are in memory.  Anyway that i would write up
in a seperate thread also considering the inputs on the current way the
patch has been made.

Regards
Ram
On Fri, Jul 19, 2013 at 4:32 PM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Like Ted and St.Ack, I read all of this with a great interest and
> everything looked good to me.
>
> My only concern will be performance wise.  Even if tags are disabled, di
> you forsee some performances impacts because everything will now need to be
> tag aware? Based on your details, I think it will be, but very minimal, or
> almost invisible, correct?
>
> Also, for migrations from v2 to v3, if v3 is activated, that will be simply
> done when HFilea will be written, correct? So not really any migration
> process required?
>
> JM
> Le 2013-07-19 01:13, "Stack" <[EMAIL PROTECTED]> a écrit :
>
> > On Thu, Jul 18, 2013 at 10:14 AM, ramkrishna vasudevan <
> > [EMAIL PROTECTED]> wrote:
> > ...
> >
> > >  We can avoid several problems with HFile V2 internals, and backwards
> > > compatibility concerns, and allow for working tags support with no
> > > performance impact and low risk to all HBase users who do not want tag
> > > support, while still allowing for inline tags capabilities in a
> shipping
> > > version of HBase, by introducing this in a new V3 version for HFile.
> > >
> > >
> > This seems like a good tactic to me.  HFileV2 has the current KV format
> > hard-coded all over and trying to 'fix' this would probably take a bunch
> of
> > effort and would jeopardize current workings.
> >
> > ....
> >
> > >
> > >  We have been working on this and will have a clean patch with good
> > amount
> > > of testing in time for 0.96.
> > >
> > >
> > I'd think that your moving into a green field by doing an hfilev3 would
> > make it so your work could run independent of 0.96 timeline; i.e. it
> could
> > come in post 0.96?
> >
> > What sort of changes do you foresee necessary in core to support cell
> > codecs?  Between rpc and hfilev3?
> >
> > Thanks Ram,
> > St.Ack
> >
>