Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Data types stage 1 is ready for reivew


Copy link to this message
-
Re: Data types stage 1 is ready for reivew
Nick Dimiduk 2013-08-08, 15:36
An extra week of reviews and comments and updates is behind us. Here's a
summary of the high-level changes.

 - Matt's ByteRange class has been refactored into an interface. That
interface is extended to PositionedByteRange that provides the API goodness
many of you desired. HBASE-9091.
 - The encoding OrderedBytes methods all operate over PositionedByteRange
now. Implementations are largely unchanged. Docstrings have been tightened
up a fair bit. HBASE-8201.
 - The data type API also operates over PositionedByteRange. For all the
data type implementations built on o.a.h.h.u.Bytes, the name "Legacy" is
replaced with "Raw." The FixedLengthWrapper class now provides a method to
inspect its length. HBASE-8693.
 - All remaining issues are outside of the critical scope of on-disk
formats and user-level APIs, and are deferred to follow-up JIRAs.

Thanks,
Nick

On Sat, Aug 3, 2013 at 11:24 AM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> This is my preferred approach. Many users don't want to have any types on
> the data because they might already have their own format, or just because
> they don't need it.
>
> So we should not enforce clients to use it.
>
> JM
>
> 2013/8/3 Nick Dimiduk <[EMAIL PROTECTED]>
>
> > On Sat, Aug 3, 2013 at 10:33 AM, Michael Segel <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > How do you plan on enforcing data types within the engine?
> > >
> >
> > Data types, at this point, are a client-side feature, not enforced by the
> > engine in anyway.
> >
> >  On Aug 1, 2013, at 10:57 PM, Matt Corgan <[EMAIL PROTECTED]> wrote:
> > >
> > > > Looks great to me.  Without the strict dependencies on hadoop or
> hbase
> > > > it'll be easy to pull into its own standalone module or new project
> if
> > > > there's demand.
> > > >
> > > >
> > > > On Thu, Aug 1, 2013 at 6:30 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > >> Finally-for-real-this-time patches posted. I'll take your +1's any
> > time
> > > now
> > > >> ;)
> > > >>
> > > >> Thanks,
> > > >> Nick
> > > >>
> > > >> On Wed, Jul 24, 2013 at 11:27 AM, Nick Dimiduk <[EMAIL PROTECTED]>
> > > wrote:
> > > >>
> > > >>> Hi everyone,
> > > >>>
> > > >>> As of yesterday, I've posted "final" patched on both HBASE-8201<
> > > >> https://issues.apache.org/jira/browse/HBASE-8201>and
> > > >>> HBASE-8693 <https://issues.apache.org/jira/browse/HBASE-8693>. The
> > > >> former
> > > >>> specifies on-disk format and the latter is the user-facing API. If
> > > you've
> > > >>> already left me a review, thank you; please have another look at
> > these
> > > >>> patches. If you have an opinion here and haven't voiced it, we're
> > > >>> approaching the "forever hold your peace" part of the ceremony.
> > > >>>
> > > >>> Thanks,
> > > >>> Nick
> > > >>>
> > > >>>
> > > >>> On Wed, Jul 17, 2013 at 9:33 AM, Nick Dimiduk <[EMAIL PROTECTED]>
> > > >> wrote:
> > > >>>
> > > >>>> Thanks for having a look. If you don't mind terribly, I responded
> to
> > > >> your
> > > >>>> comments on JIRA [0].
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Nick
> > > >>>>
> > > >>>> [0]:
> > > https://issues.apache.org/jira/browse/HBASE-8693#comment-13711250
> > > >>>>
> > > >>>>
> > > >>>> On Wed, Jul 17, 2013 at 1:42 AM, Matteo Bertozzi <
> > > >> [EMAIL PROTECTED]
> > > >>>>> wrote:
> > > >>>>
> > > >>>>> I was looking at the HBASE-8693 patch, and looks good to me for
> the
> > > >>>>> primitive types.
> > > >>>>> but I can't see how do you plan to evolve stuff like the struct.
> > > >>>>> By "evolve" I mean add/remove fields, or just query it with a
> > subset
> > > of
> > > >>>>> fields.
> > > >>>>> the fields don't have an id, and on read you must specify all of
> > them
> > > >> in
> > > >>>>> the same order as you've used for write.
> > > >>>>> (but maybe is just an immutable/fixed list of fields, and I'm ok
> > with
> > > >>>>> just
> > > >>>>> adding that info to the comment on top of the class)
> > > >>>>>
> > > >>>>>
> > > >>>>> Matteo
> > > >>>>>
> > > >>