Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Advice on setting default HBase table mapping attributes within Apache Gora


Copy link to this message
-
Re: Advice on setting default HBase table mapping attributes within Apache Gora
The following filters are in 0.94 code base as well:

./src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java
./src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java

On Wed, May 1, 2013 at 12:03 PM, <[EMAIL PROTECTED]> wrote:

> Hi,
>
> As far as I remember, there were attempts to add filtering on hbase side
> to nutch-2.x  commands, which could use SingleColumnValue filters that are
> available in hbase-0.95. So, I think it is advisable to upgrade hbase in
> gora to this version.
>
>
> Thanks.
> Alex.
>
>
>
>
>
> -----Original Message-----
> From: Ted Yu <[EMAIL PROTECTED]>
> To: user <[EMAIL PROTECTED]>
> Sent: Wed, May 1, 2013 11:53 am
> Subject: Re: Advice on setting default HBase table mapping attributes
> within Apache Gora
>
>
> 0.90.x code base is no longer actively maintained.
>
> Looking forward to the upgrade of HBase in Gora.
>
> On Wed, May 1, 2013 at 11:49 AM, Lewis John Mcgibbney <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Ted,
> > Thank you for reply.
> > This is where I drop a bomb... which I reservedly apologize for, I should
> > have dropped in original email.
> > We currently pull 0.90.4 maven artifact within Gora trunk!
> > We plan to upgrade to 0.94.X [0] after our next release (next few weeks)
> > Thanks Ted
> > [0] https://issues.apache.org/jira/browse/GORA-201
> >
> >
> > On Wed, May 1, 2013 at 11:45 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > What version of HBase are you using ?
> > >
> > > Assuming it is 0.94.x, you can find the default values
> > > in src/main/resources/hbase-default.xml
> > > e.g.
> > >   <property>
> > >     <name>hfile.block.cache.size</name>
> > >     <value>0.25</value>
> > >     <description>
> > >         Percentage of maximum heap (-Xmx setting) to allocate to block
> > > cache
> > >         used by HFile/StoreFile. Default of 0.25 means allocate 25%.
> > >         Set to 0 to disable but it's not recommended.
> > >     </description>
> > >   </property>
> > >
> > > Cheers
> > >
> > > On Wed, May 1, 2013 at 11:40 AM, Lewis John Mcgibbney <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > Hi,
> > > >
> > > > Currently in Gora, we support the following table attributes, which
> we
> > > > specify when mapping data into HBase;
> > > > compression, blockCache, blockSize, bloomFilter, maxVersions,
> > timeToLive,
> > > > inMemory .
> > > > These expand to the following
> > > >
> > > >  HColumnDescriptor columnDescriptor = getOrCreateFamily(familyName,
> > > > families);
> > > >
> > > >       if(compression != null)
> > > >
> > > > columnDescriptor.setCompressionType(Algorithm.valueOf(compression));
> > > >       if(blockCache != null)
> > > >
> > > >
> > columnDescriptor.setBlockCacheEnabled(Boolean.parseBoolean(blockCache));
> > > >       if(blockSize != null)
> > > >         columnDescriptor.setBlocksize(Integer.parseInt(blockSize));
> > > >       if(bloomFilter != null)
> > > >
> > > > columnDescriptor.setBloomFilterType(BloomType.valueOf(bloomFilter));
> > > >       if(maxVersions != null)
> > > >
> columnDescriptor.setMaxVersions(Integer.parseInt(maxVersions));
> > > >       if(timeToLive != null)
> > > >         columnDescriptor.setTimeToLive(Integer.parseInt(timeToLive));
> > > >       if(inMemory != null)
> > > >         columnDescriptor.setInMemory(Boolean.parseBoolean(inMemory));
> > > >     }
> > > >
> > > > Within GORA-218[0], it is our intention to add default values for
> such
> > > > attributes and document a bit about them as well. Currently an
> example
> > of
> > > > our hbase mapping file is here [1]. As you can see, the table block
> > only
> > > > has the name attribute, the remaining ones below are missing.
> > > >
> > > > Some guidance on setting these defaults and a brief explanation of
> what
> > > > they do would be very much appreciated.
> > > >
> > > > Thank you very much in advance for anyone able to help us out on this
> > > one.
> > > >
> > > > Lewis
> > > >
> > > > [0] https://issues.apache.org/jira/browse/GORA-218