Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How to install Snappy?


Copy link to this message
-
Re: How to install Snappy?
Done. HBASE-7264 created. I have added few more details about the
steps to follow to install Snappy on HBase 0.94.x.

I don't know how to build the documentation locally, so I'm not 100%
sure about thhe XML structure. I did cut&past so it should be good...

JM

2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
> Sure I will.
>
> JM
>
> 2012/12/3, Stack <[EMAIL PROTECTED]>:
>> Any chance of an update to
>> http://hbase.apache.org/book.html#snappy.compression ?  If someone writes
>> it up, I'll stitch it in. Thanks,
>> St.Ack
>>
>>
>> On Mon, Dec 3, 2012 at 6:29 AM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> Something more about my workaround last time:
>>>
>>> I used the following steps to test my workaround:
>>>
>>> 1) cd $HBASE_HOME
>>> ./bin/hbase org.apache.hadoop.hbase.util.CompressionTest
>>> file:///tmp/testfile lzo
>>>
>>>
>>> 2) hbase shell
>>> create 't1', {NAME => 'cf1', COMPRESSION => 'LZO'}
>>>
>>>
>>> You could modify above for your test cases.
>>>
>>> Good luck
>>> Thanks
>>> AC
>>>
>>>
>>> On 3 Dec 2012, at 10:22 PM, [EMAIL PROTECTED] wrote:
>>>
>>> > Hi JM,
>>> >
>>> > I had experienced similar error when I was installing LZO compression
>>> > to
>>> RegionServers:
>>> >
>>> > Below is from my record about installing LZO:
>>> > Issue:
>>> > java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
>>> > ...
>>> > 12/11/23 19:03:14 ERROR lzo.LzoCodec: Cannot load native-lzo without
>>> native-hadoop
>>> > Exception in thread "main" java.lang.RuntimeException: native-lzo
>>> library not available
>>> > Solution:
>>> > (compiled lzo from source), then do these EXTRA steps:
>>> > 1)  cp
>>> {working_folder_of_lzo}/hadoop-lzo-master/build/native/Linux-amd64-64/lib/*
>>> /usr/local/lib/
>>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
>>> $HBASE_HOME/conf/hbase-env.sh
>>> > 3) mkdir -p $HBASE_HOME/build
>>> > 4) cp -r {working_folder_of_lzo}/hadoop-lzo-master/build/native
>>> $HBASE_HOME/build/native
>>> >
>>> >
>>> >
>>> > When looking at your email below,  I saw your log also has "The error
>>> I'm getting is java.lang.UnsatisfiedLinkError: no hadoopsnappy in
>>> java.library.path."
>>> > I think you could try these steps:
>>> > 1)  cp {working_folder_your_snappy}/build/native/Linux-amd64-64/lib/*
>>> /usr/local/lib/
>>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
>>> $HBASE_HOME/conf/hbase-env.sh
>>> > 3) mkdir -p $HBASE_HOME/build
>>> > 4) cp -r {working_folder_your_snappy}/build/native
>>> $HBASE_HOME/build/native
>>> >
>>> >
>>> > Good luck.
>>> > Thanks
>>> > AC
>>> >
>>> >
>>> >
>>> > On 3 Dec 2012, at 9:47 PM, Jean-Marc Spaggiari wrote:
>>> >
>>> >> Ok....
>>> >>
>>> >> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2
>>> >> helped
>>> >> me and my test program is now working. I'm able to load both
>>> >> libraries. Fine.
>>> >>
>>> >> But the CompressionTest is still not working.
>>> >>
>>> >> What is very strange is that:
>>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
>>> available
>>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not
>>> loaded
>>> >>
>>> >> It's available, but not loaded.
>>> >>
>>> >> But from the code:
>>> >>  static {
>>> >>    try {
>>> >>      System.loadLibrary("snappy");
>>> >>      System.loadLibrary("hadoopsnappy");
>>> >>      LOG.warn("Snappy native library is available");
>>> >>      AVAILABLE = true;
>>> >>    } catch (UnsatisfiedLinkError ex) {
>>> >>      //NOP
>>> >>    }
>>> >>    LOADED = AVAILABLE;
>>> >>    if (LOADED) {
>>> >>      LOG.info("Snappy native library loaded");
>>> >>    } else {
>>> >>      LOG.warn("Snappy native library not loaded");
>>> >>    }
>>> >>  }
>>> >> If "Snappy native library is available" is displayed, that mean
>>> >> AVAILABLE = true... And if AVAILABLE = true, then LOADED is set to
>>> >> true and Snappy native library loaded must be displayed... But it's
>>> >> not... How is this possible?
>>> >>
>>> >> I have not expected Snappy installation to be such a challenge...
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB