Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How to install Snappy?


Copy link to this message
-
Re: How to install Snappy?
Ok, I got it!!!!

I had to copy the hadoop native libs into the hbase native libs directory!

Now I get a SUCCESS when I'm doint the CompressionTest...

I'm not 100% sure that it's the only think which was missing because I
have done so many modifications in the last 3 days...

So I will start from a blank 0.94.3 jar and re-do all the steps to
make sure it's just the native libs which need to be copied.

I was close to surrender ;)

JM

2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
> Hi Kevin,
>
> Thanks for the clarification.
>
> No, it's not what I'm seeing.
>
> Here is what I'm getting:
>
> 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library is
> available
> 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library not loaded
> Exception in thread "main" java.lang.RuntimeException: native snappy
> library not available
> at
> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:123)
> at
> org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
> at
> org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
> at
> org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:264)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:739)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:127)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:118)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:101)
> at
> org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:394)
> at
> org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:108)
> at
> org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:138)
>
> The most disturbing part is this line:
> 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library is
> available
>
> Followed by this one:
> Exception in thread "main" java.lang.RuntimeException: native snappy
> library not available
>
> Is it available? Or is it not available?
>
> I looked in the code and I have no idea why the 2nd one is raised.
>
> The code I'm looking at is on the hadoop-snappy site, but the one I
> have on my server is in the hadoop the hadoop-core-1.0.3.jar file. So
> maybe that the issue and they are different?
>
> I built the hadoop-snappy-0.0.1-SNAPSHOT.jar file too. I placed it on
> the lib folder and made sure it was taken first, but still not
> working.
>
> So far I think I will stay with GZip until Snappy is integrated on the
> HBase files...
>
> JM
>
> 2012/12/3, Kevin O'dell <[EMAIL PROTECTED]>:
>> Hey JM,
>>
>>   Sorry for the quick message earlier.  I tracked down the JIRA I was
>> referring to: https://issues.apache.org/jira/browse/HBASE-7080
>>
>> Does this look like what you are seeing in Compression test?
>>
>> On Mon, Dec 3, 2012 at 9:09 AM, Kevin O'dell
>> <[EMAIL PROTECTED]>wrote:
>>
>>> There is a compression test JIRA right now.  What are you seeing?
>>>
>>>
>>> On Mon, Dec 3, 2012 at 8:47 AM, Jean-Marc Spaggiari <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Ok....
>>>>
>>>> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2 helped
>>>> me and my test program is now working. I'm able to load both
>>>> libraries. Fine.
>>>>
>>>> But the CompressionTest is still not working.
>>>>
>>>> What is very strange is that:
>>>> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
>>>> available
>>>> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not
>>>> loaded
>>>>
>>>> It's available, but not loaded.
>>>>
>>>> But from the code:
>>>>   static {
>>>>     try {
>>>>       System.loadLibrary("snappy");
>>>>       System.loadLibrary("hadoopsnappy");
>>>>       LOG.warn("Snappy native library is available");
>>>>       AVAILABLE = true;
>>>>     } catch (UnsatisfiedLinkError ex) {
>>>>       //NOP
>>>>     }
>>>>     LOADED = AVAILABLE;
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB