ramkrishna vasudevan 2013-04-17, 17:16
ramkrishna vasudevan 2013-04-18, 07:54
Stack 2013-04-19, 01:28
ramkrishna vasudevan 2013-04-19, 03:19
Matt Corgan 2013-04-21, 21:47
Nick Dimiduk 2013-04-22, 00:08
Matt Corgan 2013-04-22, 00:36
ramkrishna vasudevan 2013-04-22, 03:58
Stack 2013-04-22, 19:28
Andrew Purtell 2013-04-22, 22:16
-Re: Cell Encoders and usage of Cell
ramkrishna vasudevan 2013-04-23, 06:40
>Until then, you are left w/ some sort of hack on KV.
> If all was Cell on the read/write path, I'd imagine
it'd be easy enough getting your tags in the mix.
Am checking on this also. May take some time till i finalise on the
Will discuss with Andy offline and get back on some initial version on the
changes that may be needed.
On Tue, Apr 23, 2013 at 3:46 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> On HBASE-7544 I added encryption to HFile starting with the encoder and
> decoder contexts and working outward, so I can attest it is possible to add
> features that wrap *around* KV serialization without changing the KV
> serialization itself. On trunk the block encoder stuff has been at least
> partially factored out so it is not so bad. The HFileV2 code in 0.94 is
> more of a hairball.
> To pursue an incremental development approach, one option to consider is
> making it possible to stack block encoders. Then we might be able to
> implement tag support as a block encoder without losing the ability to do
> real block encoding. Something else to undo is the inflexible enum that
> specifies block encoder types. We should use a short int signature or
> something instead. Think of treating block encoders like filters. Then
> encoder/decoder stacking could be assembled at runtime from metadata read
> out of hfile.
> On Mon, Apr 22, 2013 at 12:28 PM, Stack <[EMAIL PROTECTED]> wrote:
> > On Thu, Apr 18, 2013 at 8:19 PM, ramkrishna vasudevan <
> > [EMAIL PROTECTED]> wrote:
> > > My questions were mainly because, if i have the current code and i
> > > want to introduce tags in it, where would i do it?
> > >
> > Good question. If the read/write path currently does not support
> > access, you are in a bit of a bind.
> > It would take some effort converting all to pure Cell. Most would be
> > grunt work but there are a few tricky spots (according to a quick survey
> > done by our LarsH). If all was Cell on the read/write path, I'd imagine
> > it'd be easy enough getting your tags in the mix.
> > Until then, you are left w/ some sort of hack on KV.
> > > So if i need tags to be introduced should i start changing the HFile
> > > formats also and only then i would be getting the tags to work?
> > > What do you think here?
> > >
> > >
> > hfiles are versioned so we can up the version when we put a Cell API on
> > Might be a bit of work though since will have to bring along the
> > compressors and block encoders and currently these are hard-coded to
> > KV and in particular KVs current serialization.
> > St.Ack
> Best regards,
> - Andy
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)