Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - RE: Compression LZO class not found issue in Hadoop-2.2.0


Copy link to this message
-
Re: Compression LZO class not found issue in Hadoop-2.2.0
shashwat shriparv 2013-12-10, 10:38
Set the class path to where hadoop*Lzo jar file is, and then try..

*Thanks & Regards    *


Shashwat Shriparv

On Tue, Dec 10, 2013 at 4:00 PM, Vinayakumar B <[EMAIL PROTECTED]>wrote:

>  Hi Viswa,
>
>
>
> Sorry for the late reply,
>
>
>
> Have you restarted NodeManagers after copying the lzo jars to lib?
>
>
>
> Thanks and Regards,
>
> Vinayakumar B
>
> *From:* Viswanathan J [mailto:[EMAIL PROTECTED]]
> *Sent:* 06 December 2013 23:32
> *To:* [EMAIL PROTECTED]
> *Subject:* Compression LZO class not found issue in Hadoop-2.2.0
>
>
>
> Hi Team,
>
> Have added the property in mapred/core site xml and copied the hadoop lzo
> jar in hadoop lib folder. Also installed lzop,lzo-devel package in CentOS
> version.
>
> Getting the below LZO issue in Hadoop-2.2.0,
>
>  3 AttemptID:attempt_1386352289515_0001_m_000000_0 Info:Error:
> java.lang.IllegalArgumentException: Compression codec
> com.hadoop.compression.lzo.LzoCodec was not found.
>   4   at
> org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java:796)
>   5   at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1000)
>   6   at
> org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:390)
>   7   at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:79)
>   8   at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:674)
>   9   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:746)
>  10   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
>  11   at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
>  12   at java.security.AccessController.doPrivileged(Native Method)
>  13   at javax.security.auth.Subject.doAs(Subject.java:396)
>  14   at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>  15   at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
>  16 Caused by: java.lang.ClassNotFoundException: Class
> com.hadoop.compression.lzo.LzoCodec not found
>  17   at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626)
>  18   at
> org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java:794)
>  19   ... 11 more
>
>
>
> Do I need to add any other jars or property for this version?
>
> Please help. It will be very useful for production push.
>
> --
> Regards,
> Viswa.J
>