Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - How to install Snappy?


+
Jean-Marc Spaggiari 2012-12-01, 23:52
+
Jean-Marc Spaggiari 2012-12-01, 23:57
+
Jean-Marc Spaggiari 2012-12-02, 13:25
+
surfer 2012-12-03, 07:02
+
Jean-Marc Spaggiari 2012-12-03, 13:24
+
Jean-Marc Spaggiari 2012-12-03, 13:47
+
Kevin Odell 2012-12-03, 14:09
+
Kevin Odell 2012-12-03, 14:37
+
Jean-Marc Spaggiari 2012-12-03, 14:56
+
Jean-Marc Spaggiari 2012-12-03, 15:15
+
Kevin Odell 2012-12-03, 15:19
Copy link to this message
-
Re: How to install Snappy?
Jean-Marc Spaggiari 2012-12-03, 15:50
Ok. I retried and on brand new 0.94.3 installation and the only 2
things required are the libsnappy.so and the libhadoop.so files... And
it'S taking 5 minutes to install ;)

I will now deploy that all over the cluster and give snappy+0.94.3RC a try...

2012/12/3, Kevin O'dell <[EMAIL PROTECTED]>:
> Never say die!
>
> On Mon, Dec 3, 2012 at 10:15 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Ok, I got it!!!!
>>
>> I had to copy the hadoop native libs into the hbase native libs
>> directory!
>>
>> Now I get a SUCCESS when I'm doint the CompressionTest...
>>
>> I'm not 100% sure that it's the only think which was missing because I
>> have done so many modifications in the last 3 days...
>>
>> So I will start from a blank 0.94.3 jar and re-do all the steps to
>> make sure it's just the native libs which need to be copied.
>>
>> I was close to surrender ;)
>>
>> JM
>>
>> 2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
>> > Hi Kevin,
>> >
>> > Thanks for the clarification.
>> >
>> > No, it's not what I'm seeing.
>> >
>> > Here is what I'm getting:
>> >
>> > 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library is
>> > available
>> > 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library not
>> loaded
>> > Exception in thread "main" java.lang.RuntimeException: native snappy
>> > library not available
>> >       at
>> >
>> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:123)
>> >       at
>> > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
>> >       at
>> > org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
>> >       at
>> >
>> org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:264)
>> >       at
>> >
>> org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:739)
>> >       at
>> >
>> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:127)
>> >       at
>> >
>> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:118)
>> >       at
>> >
>> org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:101)
>> >       at
>> >
>> org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:394)
>> >       at
>> >
>> org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:108)
>> >       at
>> >
>> org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:138)
>> >
>> > The most disturbing part is this line:
>> > 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library is
>> > available
>> >
>> > Followed by this one:
>> > Exception in thread "main" java.lang.RuntimeException: native snappy
>> > library not available
>> >
>> > Is it available? Or is it not available?
>> >
>> > I looked in the code and I have no idea why the 2nd one is raised.
>> >
>> > The code I'm looking at is on the hadoop-snappy site, but the one I
>> > have on my server is in the hadoop the hadoop-core-1.0.3.jar file. So
>> > maybe that the issue and they are different?
>> >
>> > I built the hadoop-snappy-0.0.1-SNAPSHOT.jar file too. I placed it on
>> > the lib folder and made sure it was taken first, but still not
>> > working.
>> >
>> > So far I think I will stay with GZip until Snappy is integrated on the
>> > HBase files...
>> >
>> > JM
>> >
>> > 2012/12/3, Kevin O'dell <[EMAIL PROTECTED]>:
>> >> Hey JM,
>> >>
>> >>   Sorry for the quick message earlier.  I tracked down the JIRA I was
>> >> referring to: https://issues.apache.org/jira/browse/HBASE-7080
>> >>
>> >> Does this look like what you are seeing in Compression test?
>> >>
>> >> On Mon, Dec 3, 2012 at 9:09 AM, Kevin O'dell
>> >> <[EMAIL PROTECTED]>wrote:
>> >>
>> >>> There is a compression test JIRA right now.  What are you seeing?
>> >>>
>> >>>
>> >>> On Mon, Dec 3, 2012 at 8:47 AM, Jean-Marc Spaggiari <
>> >>> [EMAIL PROTECTED]> wrote:
>> >>>
>> >>>> Ok....
>> >>>>
>> >>>> This:
+
surfer 2012-12-04, 06:29
+
Jean-Marc Spaggiari 2012-12-04, 12:27
+
ac@...) 2012-12-03, 14:22
+
ac@...) 2012-12-03, 14:29
+
Stack 2012-12-03, 18:20
+
Jean-Marc Spaggiari 2012-12-03, 18:55
+
Jean-Marc Spaggiari 2012-12-03, 19:48
+
Stack 2012-12-03, 20:15
+
ac@...) 2012-12-02, 06:16
+
Håvard Wahl Kongsgård 2012-12-02, 23:52
+
Arati Patro 2012-12-03, 05:41
+
Mohamed Ibrahim 2012-12-03, 02:10