Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Setting up Snappy compression in Hadoop


Copy link to this message
-
Setting up Snappy compression in Hadoop
Amit Sela 2014-01-01, 16:05
Hi all,

I'm running on Hadoop 1.0.4 and I'd like to use Snappy for map output
compression.
I'm adding the configurations:

configuration.setBoolean("mapred.compress.map.output", true);
configuration.set("mapred.map.output.compression.codec",
"org.apache.hadoop.io.compress.SnappyCodec");

And I've added libsnappy.so.1 to $HADOOP_HOME/lib/native/Linux-amd64-64/

Still, all map tasks fail with "native snappy library not available".

Could anyone elaborate on how to install Snappy for Hadoop ?

Thanks,

Amit.