Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Advice on setting default HBase table mapping attributes within Apache Gora


Copy link to this message
-
Re: Advice on setting default HBase table mapping attributes within Apache Gora
Hi,

As far as I remember, there were attempts to add filtering on hbase side to nutch-2.x  commands, which could use SingleColumnValue filters that are available in hbase-0.95. So, I think it is advisable to upgrade hbase in gora to this version.
Thanks.
Alex.

 

 

-----Original Message-----
From: Ted Yu <[EMAIL PROTECTED]>
To: user <[EMAIL PROTECTED]>
Sent: Wed, May 1, 2013 11:53 am
Subject: Re: Advice on setting default HBase table mapping attributes within Apache Gora
0.90.x code base is no longer actively maintained.

Looking forward to the upgrade of HBase in Gora.

On Wed, May 1, 2013 at 11:49 AM, Lewis John Mcgibbney <
[EMAIL PROTECTED]> wrote:

> Hi Ted,
> Thank you for reply.
> This is where I drop a bomb... which I reservedly apologize for, I should
> have dropped in original email.
> We currently pull 0.90.4 maven artifact within Gora trunk!
> We plan to upgrade to 0.94.X [0] after our next release (next few weeks)
> Thanks Ted
> [0] https://issues.apache.org/jira/browse/GORA-201
>
>
> On Wed, May 1, 2013 at 11:45 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > What version of HBase are you using ?
> >
> > Assuming it is 0.94.x, you can find the default values
> > in src/main/resources/hbase-default.xml
> > e.g.
> >   <property>
> >     <name>hfile.block.cache.size</name>
> >     <value>0.25</value>
> >     <description>
> >         Percentage of maximum heap (-Xmx setting) to allocate to block
> > cache
> >         used by HFile/StoreFile. Default of 0.25 means allocate 25%.
> >         Set to 0 to disable but it's not recommended.
> >     </description>
> >   </property>
> >
> > Cheers
> >
> > On Wed, May 1, 2013 at 11:40 AM, Lewis John Mcgibbney <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi,
> > >
> > > Currently in Gora, we support the following table attributes, which we
> > > specify when mapping data into HBase;
> > > compression, blockCache, blockSize, bloomFilter, maxVersions,
> timeToLive,
> > > inMemory .
> > > These expand to the following
> > >
> > >  HColumnDescriptor columnDescriptor = getOrCreateFamily(familyName,
> > > families);
> > >
> > >       if(compression != null)
> > >
> > > columnDescriptor.setCompressionType(Algorithm.valueOf(compression));
> > >       if(blockCache != null)
> > >
> > >
> columnDescriptor.setBlockCacheEnabled(Boolean.parseBoolean(blockCache));
> > >       if(blockSize != null)
> > >         columnDescriptor.setBlocksize(Integer.parseInt(blockSize));
> > >       if(bloomFilter != null)
> > >
> > > columnDescriptor.setBloomFilterType(BloomType.valueOf(bloomFilter));
> > >       if(maxVersions != null)
> > >         columnDescriptor.setMaxVersions(Integer.parseInt(maxVersions));
> > >       if(timeToLive != null)
> > >         columnDescriptor.setTimeToLive(Integer.parseInt(timeToLive));
> > >       if(inMemory != null)
> > >         columnDescriptor.setInMemory(Boolean.parseBoolean(inMemory));
> > >     }
> > >
> > > Within GORA-218[0], it is our intention to add default values for such
> > > attributes and document a bit about them as well. Currently an example
> of
> > > our hbase mapping file is here [1]. As you can see, the table block
> only
> > > has the name attribute, the remaining ones below are missing.
> > >
> > > Some guidance on setting these defaults and a brief explanation of what
> > > they do would be very much appreciated.
> > >
> > > Thank you very much in advance for anyone able to help us out on this
> > one.
> > >
> > > Lewis
> > >
> > > [0] https://issues.apache.org/jira/browse/GORA-218
> > > [1]
> > >
> > >
> >
> http://svn.apache.org/repos/asf/gora/trunk/gora-hbase/src/test/conf/gora-hbase-mapping.xml
> > >
> > > --
> > > *Lewis*
> > >
> >
>
>
>
> --
> *Lewis*
>