Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase type support


Copy link to this message
-
Re: HBase type support
On Mon, Mar 18, 2013 at 4:54 PM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:

> +1.  I really don't want to add typing specific information into hbase core
> -- howver, having buliding blocks, plugins, and extra metadata manage it
> seems quite reasonable to me.
>
> There are many many games that can be played to encode data and enforcing
> typing at the hbase level as opposed to library. (ex: putting in structs
> that have fields with ints as opposed to having tons of cols with ints in
> them, or how opentsdb encodes time stamps, etc..).
>

I'm not proposing deep integration with core. This stuff would exist in the
client module only. The byte[] interfaces don't go away either; OpenTSDB
can continue to perform its data encoding as is. This proposal seeks to
enable less sophisticated data storage approaches, solve the common case in
a reasonable way.

-n

On Fri, Mar 15, 2013 at 10:45 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
> > I think generally we should keep HBase a byte[] based key value store.
> > What we should add to HBase are tools that would allow client side apps
> > (or libraries) to built functionality on top of plain HBase.
> >
> > Serialization that maintains a correct semantic sort order is important
> as
> > a building block, so is code that can build up correctly serialized and
> > sortable compound keys, as well as hashing algorithms.
> >
> > Where I would draw the line is adding types to HBase itself. As long as
> > one can write a client, or Filters, or Coprocessors with the tools
> provided
> > by HBase we're good. Higher level functionality can then be built of on
> top
> > of HBase.
> >
> >
> > For example, maybe we need to add better access API to the HBase WAL in
> > order to have an external library implement idempotent transactions
> (which
> > can be used to implement 2ndary indexes).
> > Maybe some other primitives have to be exposed in order to allow an
> > external library to implement full transactions.
> > Or we might need a statistics framework (such as the one that Jesse is
> > working on).
> >
> > These are all building blocks that do not presume specific access
> patterns
> > or clients, but can be used to implement them.
> >
> >
> > As usual, just my $0.02.
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> >  From: Nick Dimiduk <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Sent: Friday, March 15, 2013 10:57 AM
> > Subject: Re: HBase type support
> >
> > I'm talking about MD5, SHA1, etc. It's something explicitly mentioned
> > in HBASE-7221.
> >
> > On Fri, Mar 15, 2013 at 10:55 AM, James Taylor <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi Nick,
> > > What do you mean by "hashing algorithms"?
> > > Thanks,
> > > James
> > >
> > >
> > > On 03/15/2013 10:11 AM, Nick Dimiduk wrote:
> > >
> > >> Hi David,
> > >>
> > >> Native support for a handful of hashing algorithms has also been
> > >> discussed.
> > >> Do you think these should be supported directly, as opposed to using a
> > >> fixed-length String or fixed-length byte[]?
> > >>
> > >> Thanks,
> > >> Nick
> > >>
> > >> On Thu, Mar 14, 2013 at 9:51 AM, David Koch <[EMAIL PROTECTED]>
> > >> wrote:
> > >>
> > >>  Hi Nick,
> > >>>
> > >>> As an HBase user I would welcome this addition. In addition to the
> > >>> proposed
> > >>> list of datatypes A UUID/GUID type would also be nice to have.
> > >>>
> > >>> Regards,
> > >>>
> > >>> /David
> > >>>
> > >>>
> > >>> On Wed, Mar 13, 2013 at 5:42 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> > >>> wrote:
> > >>>
> > >>>  Hi all,
> > >>>>
> > >>>> I'd like to draw your attention to HBASE-8089. The desire is to add
> > type
> > >>>> support to HBase. There are two primary objectives: make the lives
> of
> > >>>> developers building on HBase easier, and facilitate better tools on
> > top
> > >>>>
> > >>> of
> > >>>
> > >>>> HBase. Please chime in with any feature suggestions you think we've
> > >>>>
> > >>> missed
> > >>>
> > >>>> in initial conversations.
> > >>>>
> > >>>> Thanks,