Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Compression LZO class not found issue in Hadoop-2.2.0


Copy link to this message
-
Compression LZO class not found issue in Hadoop-2.2.0
Hi Team,

Have added the property in mapred/core site xml and copied the hadoop lzo
jar in hadoop lib folder. Also installed lzop,lzo-devel package in CentOS
version.

Getting the below LZO issue in Hadoop-2.2.0,

 3 AttemptID:attempt_1386352289515_0001_m_000000_0 Info:Error:
java.lang.IllegalArgumentException: Compression codec
com.hadoop.compression.lzo.LzoCodec was not found.
  4   at
org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java:796)
  5   at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1000)
  6   at
org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:390)
  7   at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:79)
  8   at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:674)
  9   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:746)
 10   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
 11   at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
 12   at java.security.AccessController.doPrivileged(Native Method)
 13   at javax.security.auth.Subject.doAs(Subject.java:396)
 14   at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
 15   at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
 16 Caused by: java.lang.ClassNotFoundException: Class
com.hadoop.compression.lzo.LzoCodec not found
 17   at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626)
 18   at
org.apache.hadoop.mapred.JobConf.getMapOutputCompressorClass(JobConf.java:794)
 19   ... 11 more

Do I need to add any other jars or property for this version?

Please help. It will be very useful for production push.

--
Regards,
Viswa.J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB