Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Failing to get Snappy working (hours now - native snappy library not available) - Help?


Copy link to this message
-
Re: Failing to get Snappy working (hours now - native snappy library not available) - Help?
Ted 2013-04-04, 00:37
btw, if you happen to be using something like fedora... the following
easy 4 steps worked for me

- on fedora yum install snappy
- ln -s /usr/lib64/libsnappy.so.1.1.3
/opt/hbase/lib/native/Linux-amd-64-64/libsnappy.so
- ln -s /opt/hadoop/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
/opt/hbase/lib/native/Linux-amd-64-64/libhadoop.so
- test it with : bin/hbase
org.apache.hadoop.hbase.util.CompressionTest file:///tmp/snappy snappy
It might save you all the fuss with building and installing snappy if
you just use the provided OS packages (some one will probably tell me
it's the wrong version now :) )

On 4/4/13, Andrew Purtell <[EMAIL PROTECTED]> wrote:
> Yes Jeff, I agree and will take care of it. Thanks for bringing this to our
> attention.
>
> On Wednesday, April 3, 2013, Jeff Blaine wrote:
>
>> On 4/3/2013 4:42 PM, Andrew Purtell wrote:
>>
>>> Make sure the path to the libhadoop SO is on LD_LIBRARY_PATH or
>>> somewhere
>>> in /etc/ld.so.conf (and if the latter then also rebuild the ld cache
>>> with
>>> /sbin/ldconfig)
>>>
>>
>> Thanks Andrew.
>>
>> If that's required to even get things working, shouldn't the
>> documentation mention that? :(
>>
>>  On Wednesday, April 3, 2013, Jeff Blaine wrote:
>>>
>>>  Hello,
>>>>
>>>> I cannot see what is amiss with the following configuration. I have
>>>> followed, to the best of my knowledge, the instructions to a "T".
>>>>
>>>> If anyone could shed some light, I would be grateful. All HBase
>>>> hosts have the exact same configuration shown below and non-SNAPPY
>>>> table operations work fine...
>>>>
>>>> $ id
>>>> uid=30000(hadoop) gid=30000(hadoop) groups=30000(hadoop)
>>>> $ uname -i
>>>> x86_64
>>>> $ pwd
>>>> /hbase-0.92.2
>>>> $ grep HBASE_LIBRARY_PATH conf/hbase-env.sh
>>>> export HBASE_LIBRARY_PATH="/hadoop-1.****0.4/lib/native/Linux-amd64-**
>>>> 64"
>>>> $ cd /hadoop-1.0.4/lib/native/****Linux-amd64-64
>>>> $ ls -l
>>>> total 1456
>>>> -rw-rw-r-- 1 hadoop root   405228 Oct  3 05:14 libhadoop.a
>>>> -rw-rw-r-- 1 hadoop root      878 Oct  3 05:14 libhadoop.la
>>>> -rw-rw-r-- 1 hadoop root   219217 Oct  3 05:14 libhadoop.so
>>>> -rw-rw-r-- 1 hadoop root   219217 Oct  3 05:14 libhadoop.so.1
>>>> -rw-rw-r-- 1 hadoop root   219217 Oct  3 05:14 libhadoop.so.1.0.0
>>>> -rw-rw-r-- 1 hadoop ubuntu 136297 Apr  3 19:46 libsnappy.so
>>>> -rw-rw-r-- 1 hadoop ubuntu 136297 Apr  3 19:47 libsnappy.so.1
>>>> -rw-rw-r-- 1 hadoop ubuntu 136297 Apr  3 19:46 libsnappy.so.1.1.4
>>>> $ /hbase-0.92.2/bin/hbase shell
>>>> HBase Shell; enter 'help<RETURN>' for list of supported commands.
>>>> Type "exit<RETURN>" to leave the HBase Shell
>>>> Version 0.92.2, r1379292, Fri Aug 31 13:13:53 UTC 2012
>>>>
>>>> hbase(main):001:0> create 'non', 'cf'
>>>> 0 row(s) in 4.0720 seconds
>>>>
>>>> hbase(main):002:0> create 'comp', { NAME => 'foo', COMPRESSION =>
>>>> 'SNAPPY'
>>>> }
>>>> [ The command above never returns ]
>>>> ^C
>>>> $
>>>>
>>>> Logs show:
>>>>
>>>> 2013-04-03 18:07:11,992 WARN org.apache.hadoop.io.compress.**
>>>> **snappy.LoadSnappy:
>>>> Snappy native library not loaded
>>>> java.io.IOException: java.lang.RuntimeException: native snappy library
>>>> not
>>>> available
>>>>
>>>>
>>>
>>>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Ted.