Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - snappy codec


Copy link to this message
-
Re: snappy codec
Mohit Anchlia 2012-06-11, 16:26
use java lib path

On Mon, Jun 11, 2012 at 7:12 AM, GUOJUN Zhu <[EMAIL PROTECTED]>wrote:

>
> I believe that the mapred.child.env only concerns the spawned child
> process where the map is run.  You probably need to put this path into the
> system property where the map task demon runs and possibly where the job
> controller runs.
>
> Zhu, Guojun
> Modeling Sr Graduate
> 571-3824370
> [EMAIL PROTECTED]
> Financial Engineering
> Freddie Mac
>
>
>      *Marek Miglinski <[EMAIL PROTECTED]>*
>
>    06/11/2012 09:54 AM
>      Please respond to
> [EMAIL PROTECTED]
>
>    To
> "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> cc
>   Subject
> snappy codec
>
>
>
>
> Hi,
>
> I have a Clouderas CDH3u3 installed on my cluster and mapred.child.env set
> to "LD_LIBRARY_PATH=/usr/lib/hadoop-0.20/lib/native/Linux-amd64-64" (with
> libsnappy.so in the folder) in mapred-site.xml. Cloudera says that Snappy
> is included in their hadoop-0.20-native package and it is also installed on
> each of the nodes.
>
> But when I run a mapreduce task with "mapred.map.output.compression.codec"
> set to "org.apache.hadoop.io.compress.SnappyCodec" I get an exception:
>
> java.lang.RuntimeException: native snappy library not available
>
>
> Any idea why?
>
>
>
> Thanks,
> Marek M.
>
>