Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - How to install Snappy?


Copy link to this message
-
Re: How to install Snappy?
Jean-Marc Spaggiari 2012-12-03, 19:48
Done. HBASE-7264 created. I have added few more details about the
steps to follow to install Snappy on HBase 0.94.x.

I don't know how to build the documentation locally, so I'm not 100%
sure about thhe XML structure. I did cut&past so it should be good...

JM

2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
> Sure I will.
>
> JM
>
> 2012/12/3, Stack <[EMAIL PROTECTED]>:
>> Any chance of an update to
>> http://hbase.apache.org/book.html#snappy.compression ?  If someone writes
>> it up, I'll stitch it in. Thanks,
>> St.Ack
>>
>>
>> On Mon, Dec 3, 2012 at 6:29 AM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> Something more about my workaround last time:
>>>
>>> I used the following steps to test my workaround:
>>>
>>> 1) cd $HBASE_HOME
>>> ./bin/hbase org.apache.hadoop.hbase.util.CompressionTest
>>> file:///tmp/testfile lzo
>>>
>>>
>>> 2) hbase shell
>>> create 't1', {NAME => 'cf1', COMPRESSION => 'LZO'}
>>>
>>>
>>> You could modify above for your test cases.
>>>
>>> Good luck
>>> Thanks
>>> AC
>>>
>>>
>>> On 3 Dec 2012, at 10:22 PM, [EMAIL PROTECTED] wrote:
>>>
>>> > Hi JM,
>>> >
>>> > I had experienced similar error when I was installing LZO compression
>>> > to
>>> RegionServers:
>>> >
>>> > Below is from my record about installing LZO:
>>> > Issue:
>>> > java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
>>> > ...
>>> > 12/11/23 19:03:14 ERROR lzo.LzoCodec: Cannot load native-lzo without
>>> native-hadoop
>>> > Exception in thread "main" java.lang.RuntimeException: native-lzo
>>> library not available
>>> > Solution:
>>> > (compiled lzo from source), then do these EXTRA steps:
>>> > 1)  cp
>>> {working_folder_of_lzo}/hadoop-lzo-master/build/native/Linux-amd64-64/lib/*
>>> /usr/local/lib/
>>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
>>> $HBASE_HOME/conf/hbase-env.sh
>>> > 3) mkdir -p $HBASE_HOME/build
>>> > 4) cp -r {working_folder_of_lzo}/hadoop-lzo-master/build/native
>>> $HBASE_HOME/build/native
>>> >
>>> >
>>> >
>>> > When looking at your email below,  I saw your log also has "The error
>>> I'm getting is java.lang.UnsatisfiedLinkError: no hadoopsnappy in
>>> java.library.path."
>>> > I think you could try these steps:
>>> > 1)  cp {working_folder_your_snappy}/build/native/Linux-amd64-64/lib/*
>>> /usr/local/lib/
>>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
>>> $HBASE_HOME/conf/hbase-env.sh
>>> > 3) mkdir -p $HBASE_HOME/build
>>> > 4) cp -r {working_folder_your_snappy}/build/native
>>> $HBASE_HOME/build/native
>>> >
>>> >
>>> > Good luck.
>>> > Thanks
>>> > AC
>>> >
>>> >
>>> >
>>> > On 3 Dec 2012, at 9:47 PM, Jean-Marc Spaggiari wrote:
>>> >
>>> >> Ok....
>>> >>
>>> >> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2
>>> >> helped
>>> >> me and my test program is now working. I'm able to load both
>>> >> libraries. Fine.
>>> >>
>>> >> But the CompressionTest is still not working.
>>> >>
>>> >> What is very strange is that:
>>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
>>> available
>>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not
>>> loaded
>>> >>
>>> >> It's available, but not loaded.
>>> >>
>>> >> But from the code:
>>> >>  static {
>>> >>    try {
>>> >>      System.loadLibrary("snappy");
>>> >>      System.loadLibrary("hadoopsnappy");
>>> >>      LOG.warn("Snappy native library is available");
>>> >>      AVAILABLE = true;
>>> >>    } catch (UnsatisfiedLinkError ex) {
>>> >>      //NOP
>>> >>    }
>>> >>    LOADED = AVAILABLE;
>>> >>    if (LOADED) {
>>> >>      LOG.info("Snappy native library loaded");
>>> >>    } else {
>>> >>      LOG.warn("Snappy native library not loaded");
>>> >>    }
>>> >>  }
>>> >> If "Snappy native library is available" is displayed, that mean
>>> >> AVAILABLE = true... And if AVAILABLE = true, then LOADED is set to
>>> >> true and Snappy native library loaded must be displayed... But it's
>>> >> not... How is this possible?
>>> >>
>>> >> I have not expected Snappy installation to be such a challenge...