Hadoop, mail # user - Re: error:Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzopCodec

error:Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzopCodec

Alex Luya 2010-07-28, 14:42

Hello：
   I got source code from http://github.com/kevinweil/hadoop-lzo，compiled
them successfully,and then
1，copy hadoop-lzo-0.4.4.jar to directory:$HADOOP_HOME/lib of each master and
slave
2，Copy all files under directory:../Linux-amd64-64/lib to directory:
$HADDOOP_HOME/lib/native/Linux-amd64-64 of each master and slave
3,and upload a file:test.lzo to HDFS
4，then run：hadoop jar $HADOOP_HOME/lib/hadoop-lzo-0.4.4.jar
com.hadoop.compression.lzo.DistributedLzoIndexer test.lzo to test

got errors：
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10/07/20 22:37:37 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
10/07/20 22:37:37 INFO lzo.LzoCodec: Successfully loaded & initialized native-
lzo library [hadoop-lzo rev 5c25e0073d3dae9ace4bd9eba72e4dc43650c646]
##########^_^^_^^_^^_^^_^^_^##################
(I think this says all native library got loaded successfully)
################################
10/07/20 22:37:37 INFO lzo.DistributedLzoIndexer: Adding LZO file target.lz:o
to indexing list (no index currently exists)
...
attempt_201007202234_0001_m_000000_0, Status : FAILED
java.lang.IllegalArgumentException: Compression codec
com.hadoop.compression.lzo.LzopCodec
   not found.
   at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:96)
   at
org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
   at
com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:48)
   at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418)
   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620)
   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
   at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException:
com.hadoop.compression.lzo.LzopCodec

   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
   at java.lang.Class.forName0(Native Method)
   at java.lang.Class.forName(Class.java:247)
   at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
   at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:89)
   ... 6 more
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

There is a installation instruction in this
link:http://github.com/kevinweil/hadoop-lzo，it says other configurings are
needed ：

Once the libs are built and installed, you may want to add them to the class
paths and library paths. That is, in hadoop-env.sh, set

   (1)export HADOOP_CLASSPATH=/path/to/your/hadoop-lzo-lib.jar

Question:I have copied hadoop-lzo-0.4.4.jar to $HADOOP_HOME/lib,
，should I do set this entry like this again? actually, after I add this:
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HBASE_HOME/hbase-0.20.4.jar:
$HABSE_HOME/config:$ZOOKEEPER_HOME/zookeeper-3.3.1.jar:$HADOOP_HOME/lib
/hadoop-lzo-0.4.4.jar，redo 1-4 as above,same problem as before,so:
how can I
get hadoop to load hadoop-lzo-0.4.4.jar？）
   (2),export JAVA_LIBRARY_PATH=/path/to/hadoop-lzo-native-
libs:/path/to/standard-hadoop-native-libs
   Note that there seems to be a bug in /path/to/hadoop/bin/hadoop; comment
out the line
   (3)JAVA_LIBRARY_PATH=''
Question：since native library got loaded successfully，aren't these
operation(2)(3) needed？
－－－－－－－－��－－－－－－－��－－－－－－－－－－－－－－－－��－－－－－－－��－－－－－
I am using hadoop 0.20.2
core-site.xml
<configuration>
   <property>
   <name>fs.default.name</name>
   <value>hdfs://hadoop:8020</value>
   </property>
   <property>
   <name>hadoop.tmp.dir</name>
   <value>/home/hadoop/tmp</value>
   </property>

   <property>
   <name>io.compression.codecs</name>

<value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec
   </value>
   </property>
   <property>
   <name>io.compression.codec.lzo.class</name>
   <value>com.hadoop.compression.lzo.LzoCodec</value>
   </property>
</configuration>

mapred-site.xml
<configuration>
   <property>
   <name>mapred.job.tracker</name>
   <value>AlexLuya:9001</value>
   </property>
   <property>
   <name>mapred.tasktracker.reduce.tasks.maximum</name>
   <value>1</value>
   </property>
   <property>
   <name>mapred.tasktracker.map.tasks.maximum</name>
   <value>1</value>
   </property>
   <property>
   <name>mapred.local.dir</name>
   <value>/home/alex/hadoop/mapred/local</value>
   </property>
   <property>
   <name>mapred.system.dir</name>
   <value>/tmp/hadoop/mapred/system</value>
   </property>
   <property>
   <name>mapreduce.map.output.compress</name>
   <value>true</value>
   </property>
   <property>
   <name>mapreduce.map.output.compress.codec</name>
   <value>com.hadoop.compression.lzo.LzoCodec</value>
   </property>
</configuration>

hadoop-env.sh
# Set Hadoop-specifi

Re: error:Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzopCodec

Alex Kozlov 2010-07-28, 23:34

Hi Alex,

There seems to be a problem with your configuration. Can you try and check
on which node the attempt_201007202234_0001_m_000000_0 was run? Go to this
machine and check the config files and the lib subdirectory (for the
presence of the correct configuration and the hadoop-lzo-0.4.4.jar).
Restart the TT and using 'ps -aef | grep -i tasktracker' check that the
hadoop-lzo-0.4.4.jar is in the classpath.

I have a strong suspicion you have stray config files:
com.hadoop.compression.lzo.LzopCodec is not mentioned in the ones you
provided...

Alex K

On Wed, Jul 28, 2010 at 7:42 AM, Alex Luya <[EMAIL PROTECTED]> wrote:

> Hello：
> I got source code from http://github.com/kevinweil/hadoop-lzo，compiled
> them successfully,and then
> 1，copy hadoop-lzo-0.4.4.jar to directory:$HADOOP_HOME/lib of each master
> and
> slave
> 2，Copy all files under directory:../Linux-amd64-64/lib to directory:
> $HADDOOP_HOME/lib/native/Linux-amd64-64 of each master and slave
> 3,and upload a file:test.lzo to HDFS
> 4，then run：hadoop jar $HADOOP_HOME/lib/hadoop-lzo-0.4.4.jar
> com.hadoop.compression.lzo.DistributedLzoIndexer test.lzo to test
>
> got errors：
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> 10/07/20 22:37:37 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 10/07/20 22:37:37 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-
> lzo library [hadoop-lzo rev 5c25e0073d3dae9ace4bd9eba72e4dc43650c646]
> ##########^_^^_^^_^^_^^_^^_^##################
> (I think this says all native library got loaded successfully)
> ################################
> 10/07/20 22:37:37 INFO lzo.DistributedLzoIndexer: Adding LZO file
> target.lz:o
> to indexing list (no index currently exists)
> ...
> attempt_201007202234_0001_m_000000_0, Status : FAILED
> java.lang.IllegalArgumentException: Compression codec
> com.hadoop.compression.lzo.LzopCodec
> not found.
> at
>
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:96)
> at
>
> org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
> at
>
> com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:48)
> at
>
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.ClassNotFoundException:
> com.hadoop.compression.lzo.LzopCodec
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
> at
>
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:89)
> ... 6 more
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>
> There is a installation instruction in this
> link:http://github.com/kevinweil/hadoop-lzo，it says other configurings are
> needed ：
>
> Once the libs are built and installed, you may want to add them to the
> class
> paths and library paths. That is, in hadoop-env.sh, set
>
> (1)export HADOOP_CLASSPATH=/path/to/your/hadoop-lzo-lib.jar

Re: error:Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzopCodec

Alex Luya 2010-07-29, 14:57

Hi,

Run:ps -aef | grep -i tasktracker
I got this:
-------------------------------------------------------------------------------------------------
alex 2425 1 0 22:34 ? 00:00:05
/usr/local/hadoop/jdk1.6.0_20/bin/java -Xmx200m -
Dhadoop.log.dir=/usr/local/hadoop/hadoop-0.20.2/bin/../logs -
Dhadoop.log.file=hadoop-alex-tasktracker-Hadoop-03.log -
Dhadoop.home.dir=/usr/local/hadoop/hadoop-0.20.2/bin/.. -Dhadoop.id.str=alex -
Dhadoop.root.logger=INFO,DRFA -
Djava.library.path=/usr/local/hadoop/hadoop-0.20.2/bin/../lib/native/Linux-
amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath
/usr/local/hadoop/hadoop-0.20.2/bin/../conf:/usr/local/hadoop/jdk1.6.0_20/lib/tools.jar:/usr/local/hadoop/hadoop-0.20.2/bin/..:/usr/local/hadoop/hadoop-0.20.2/bin/../hadoop-0.20.2-
core.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
cli-1.2.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
codec-1.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
el-1.0.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
httpclient-3.0.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
logging-1.0.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-logging-
api-1.0.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
net-1.4.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/core-3.1.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hadoop-
lzo-0.4.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jasper-
compiler-5.5.12.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jasper-
runtime-5.5.12.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jetty-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jetty-
util-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/junit-3.8.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/kfs-0.2.2.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/log4j-1.2.15.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/mockito-
all-1.8.0.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/oro-2.0.8.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/servlet-
api-2.5-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/slf4j-
api-1.4.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/slf4j-
log4j12-1.4.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/xmlenc-0.52.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-
api-2.1.jar org.apache.hadoop.mapred.TaskTracker
alex 2609 1231 0 22:51 pts/0 00:00:00 grep --color=auto -i
tasktracker
-----------------------------------------------------------------------------------------------------------------------------------------------------
see this:
:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hadoop-lzo-0.4.4.jar

"hadoop-lzo-0.4.4.jar" got presented,but question is why directory structure
will like this "/bin/../lib/hadoop-lzo-0.4.4.jar",it should be
"/usr/local/hadoop/hadoop-0.20.2/lib/hadoop-lzo-0.4.4.jar",is this normal?
On Thursday, July 29, 2010 07:34:45 am Alex Kozlov wrote:
> Hi Alex,
>
> There seems to be a problem with your configuration. Can you try and check
> on which node the attempt_201007202234_0001_m_000000_0 was run? Go to this
> machine and check the config files and the lib subdirectory (for the
> presence of the correct configuration and the hadoop-lzo-0.4.4.jar).
> Restart the TT and using 'ps -aef | grep -i tasktracker' check that the
> hadoop-lzo-0.4.4.jar is in the classpath.
>
> I have a strong suspicion you have stray config files:
> com.hadoop.compression.lzo.LzopCodec is not mentioned in the ones you
> provided...
>
> Alex K
>
> On Wed, Jul 28, 2010 at 7:42 AM, Alex Luya <[EMAIL PROTECTED]> wrote:
> > Hello：
> >
> > I got source code from http://github.com/kevinweil/hadoop-lzo��compiled
> >
> > them successfully,and then
> > 1，copy hadoop-lzo-0.4.4.jar to directory:$HADOOP_HOME/lib of each master
> > and

Re: error:Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzopCodec

Ted Yu 2010-07-29, 16:11

Yes.

On Thu, Jul 29, 2010 at 7:57 AM, Alex Luya <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Run:ps -aef | grep -i tasktracker
> I got this:
>
> -------------------------------------------------------------------------------------------------
> alex 2425 1 0 22:34 ? 00:00:05
> /usr/local/hadoop/jdk1.6.0_20/bin/java -Xmx200m -
> Dhadoop.log.dir=/usr/local/hadoop/hadoop-0.20.2/bin/../logs -
> Dhadoop.log.file=hadoop-alex-tasktracker-Hadoop-03.log -
> Dhadoop.home.dir=/usr/local/hadoop/hadoop-0.20.2/bin/..
> -Dhadoop.id.str=alex -
> Dhadoop.root.logger=INFO,DRFA -
> Djava.library.path=/usr/local/hadoop/hadoop-0.20.2/bin/../lib/native/Linux-
> amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath
>
> /usr/local/hadoop/hadoop-0.20.2/bin/../conf:/usr/local/hadoop/jdk1.6.0_20/lib/tools.jar:/usr/local/hadoop/hadoop-0.20.2/bin/..:/usr/local/hadoop/hadoop-0.20.2/bin/../hadoop-0.20.2-
> core.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
> cli-1.2.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
> codec-1.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
> el-1.0.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
> httpclient-3.0.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
>
> logging-1.0.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-logging-
> api-1.0.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
>
> net-1.4.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/core-3.1.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hadoop-
>
> lzo-0.4.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jasper-
> compiler-5.5.12.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jasper-
>
> runtime-5.5.12.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jetty-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jetty-
>
> util-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/junit-3.8.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/kfs-0.2.2.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/log4j-1.2.15.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/mockito-
>
> all-1.8.0.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/oro-2.0.8.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/servlet-
> api-2.5-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/slf4j-
> api-1.4.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/slf4j-
>
> log4j12-1.4.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/xmlenc-0.52.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-
> api-2.1.jar org.apache.hadoop.mapred.TaskTracker
> alex 2609 1231 0 22:51 pts/0 00:00:00 grep --color=auto -i
> tasktracker
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------
>
>
> see this:
> :/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hadoop-lzo-0.4.4.jar
>
> "hadoop-lzo-0.4.4.jar" got presented,but question is why directory
> structure
> will like this "/bin/../lib/hadoop-lzo-0.4.4.jar",it should be
> "/usr/local/hadoop/hadoop-0.20.2/lib/hadoop-lzo-0.4.4.jar",is this normal?
>
>
> On Thursday, July 29, 2010 07:34:45 am Alex Kozlov wrote:
> > Hi Alex,
> >
> > There seems to be a problem with your configuration. Can you try and
> check
> > on which node the attempt_201007202234_0001_m_000000_0 was run? Go to
> this
> > machine and check the config files and the lib subdirectory (for the
> > presence of the correct configuration and the hadoop-lzo-0.4.4.jar).
> > Restart the TT and using 'ps -aef | grep -i tasktracker' check that the
> > hadoop-lzo-0.4.4.jar is in the classpath.
> >
> > I have a strong suspicion you have stray config files:
> > com.hadoop.compression.lzo.LzopCodec is not mentioned in the ones you
> > provided...
> >
> > Alex K
> >
> > On Wed, Jul 28, 2010 at 7:42 AM, Alex Luya <[EMAIL PROTECTED]>
> wrote:

Re: error:Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzopCodec

Sudhir Vallamkondu 2010-08-04, 23:54

Couple of things you can try.

Did you restart the task trackers after installing the lzo libs? You can
check if the lzo lib is in the classpath or not by doing

ps auxw | grep tasktracker | grep lzo | wc –l

The above returns 1 if classpath contains the lzo lib

There is a local LZO indexer that you can run on each tasktracker to check
it works right or not

hadoop jar $HADOOP_HOME/lib/hadoop-lzo-0.4.4.jar
com.hadoop.compression.lzo.LzoIndexer test.lzo
On Wed, Jul 28, 2010 at 7:42 AM, Alex Luya <[EMAIL PROTECTED]> wrote:

> Hello：
> I got source code from http://github.com/kevinweil/hadoop-lzo，compiled
> them successfully,and then
> 1，copy hadoop-lzo-0.4.4.jar to directory:$HADOOP_HOME/lib of each master
> and
> slave
> 2，Copy all files under directory:../Linux-amd64-64/lib to directory:
> $HADDOOP_HOME/lib/native/Linux-amd64-64 of each master and slave
> 3,and upload a file:test.lzo to HDFS
> 4，then run：hadoop jar $HADOOP_HOME/lib/hadoop-lzo-0.4.4.jar
> com.hadoop.compression.lzo.DistributedLzoIndexer test.lzo to test
>
> got errors：
>
>
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
-------------
> 10/07/20 22:37:37 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
> 10/07/20 22:37:37 INFO lzo.LzoCodec: Successfully loaded & initialized
> native-
> lzo library [hadoop-lzo rev 5c25e0073d3dae9ace4bd9eba72e4dc43650c646]
> ##########^_^^_^^_^^_^^_^^_^##################
> (I think this says all native library got loaded successfully)
> ################################
> 10/07/20 22:37:37 INFO lzo.DistributedLzoIndexer: Adding LZO file
> target.lz:o
> to indexing list (no index currently exists)
> ...
> attempt_201007202234_0001_m_000000_0, Status : FAILED
> java.lang.IllegalArgumentException: Compression codec
> com.hadoop.compression.lzo.LzopCodec
> not found.
> at
>
>
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(Compressio
nCodecFactory.java:96)
> at
>
>
org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFac
tory.java:134)
> at
>
>
com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:4
8)
> at
>
>
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java
:418)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.ClassNotFoundException:
> com.hadoop.compression.lzo.LzopCodec
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
> at
>
>
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(Compressio
nCodecFactory.java:89)
> ... 6 more
>
>
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------
>
>
> There is a installation instruction in this
> link:http://github.com/kevinweil/hadoop-lzo，it says other configurings are
> needed ：
>
> Once the libs are built and installed, you may want to add them to the
> class
> paths and library paths. That is, in hadoop-env.sh, set
>
> (1)export HADOOP_CLASSPATH=/path/to/your/hadoop-lzo-lib.jar
>
> Question:I have copied hadoop-lzo-0.4.4.jar to $HADOOP_HOME/lib,
苤苤苤苤苤苤苤苤苤苤苤苤�
－－－－－－－－－－－－�
苤苤苤苤苤苤苤苤苤�> I am using hadoop 0.20.2
<value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.Def
aultCodec,org.apache.hadoop.io.compress.BZip2Codec,com.hadoop.compression.lzo.Lz
oCodec,com.hadoop.compression.lzo.LzopCodec
$HABSE_HOME/config:$ZOOKEEPER_HOME/zookeeper-3.3.1.jar:$HADOOP_HOME/lib/hadoop-
iCrossing Privileged and Confidential Information
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information of iCrossing. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.