Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> problem with hadoop-snappy


+
alxsss@... 2012-11-06, 01:10
+
Binglin Chang 2012-11-06, 02:06
+
alxsss@... 2012-11-06, 21:30
+
Binglin Chang 2012-11-07, 01:34
+
alxsss@... 2012-11-07, 01:42
+
Arpit Gupta 2012-11-07, 01:55
Copy link to this message
-
Re: problem with hadoop-snappy
I think what you're hitting is probably
https://issues.apache.org/jira/browse/HADOOP-8756.  It was fixed in
branch-2 and branch-3, but not in the old releases.

If you want to work around the problem, try explicitly setting
LD_LIBRARY_PATH to include the directory that contains snappy.

cheers,
Colin
On Tue, Nov 6, 2012 at 1:30 PM,  <[EMAIL PROTECTED]> wrote:
> When I used hadoop-1.0.4  without the installation of hadoop-snappy I saw in
> the logs
>
>  WARN snappy.LoadSnappy: Snappy native library not loaded
>
> So, it was my understanding that snappy is not included with hadoop-1.0.4.
> After removing the libs I installed, I see again the same warning.
>
> Any ideas how to fix this issue?
>
> Thanks.
> Alex.
>
>
>
>
> -----Original Message-----
> From: Binglin Chang <[EMAIL PROTECTED]>
> To: user <[EMAIL PROTECTED]>
> Sent: Mon, Nov 5, 2012 6:07 pm
> Subject: Re: problem with hadoop-snappy
>
> I think hadoop-1.0.4 already have snappy included, you should not using
> other third party libraries.
>
>
>
> On Tue, Nov 6, 2012 at 9:10 AM, <[EMAIL PROTECTED]> wrote:
>>
>> Hello,
>>
>> I use hadoop-1.0.4  I have followed  instruction to install hadoop-snappy
>> at
>> http://code.google.com/p/hadoop-snappy/
>>
>> When I run a mapred job I see
>>
>> FATAL org.apache.hadoop.mapred.TaskTracker: Task:
>> attempt_201211051656_0002_m_000000_3 - Killed :
>> org.apache.hadoop.io.compress.snappy.SnappyCompressor.compressBytesDirect()I
>>
>> in task tracker log and the job fails.
>>
>> Any ideas how to fix this issue.
>>
>> Thanks in advance.
>> Alex.
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB