Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: how to get lzo library library loaded?(error :Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec)


Copy link to this message
-
Re: how to get lzo library library loaded?(error :Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec)
Alex Luya 2010-08-16, 12:06
Thank you,
I followed your guide:http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201006.mbox/<AANLkTileo-
[EMAIL PROTECTED]>#0;,same error as before,when I run:

 ps -eaf|grep tasktracker|grep classpath
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
alex      1743     1  0 19:31 ?        00:00:04 /usr/local/hadoop/jdk1.6.0_21/bin/java -Xmx1000m -
Dhadoop.log.dir=/usr/local/hadoop/hadoop-0.20.2/bin/../logs -Dhadoop.log.file=hadoop-alex-tasktracker-Hadoop-01.log -
Dhadoop.home.dir=/usr/local/hadoop/hadoop-0.20.2/bin/.. -Dhadoop.id.str=alex -Dhadoop.root.logger=INFO,DRFA -
Djava.library.path=/usr/local/hadoop/hadoop-0.20.2/lib/native/Linux-amd64-64:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/native/Linux-amd64-64 -
Dhadoop.policy.file=hadoop-policy.xml -classpath
/usr/local/hadoop/hadoop-0.20.2/conf:/usr/local/hadoop/jdk1.6.0_21/lib/tools.jar:/usr/local/hadoop/hadoop-0.20.2/bin/..:/usr/local/hadoop/hadoop-0.20.2/bin/../hadoop-
core-0.20.2+320.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-cli-1.2.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
codec-1.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-el-1.0.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
httpclient-3.0.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-logging-1.0.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-logging-
api-1.0.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/commons-
net-1.4.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/core-3.1.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hadoop-
fairscheduler-0.20.2+320.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hadoop-lzo-0.4.4.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hadoop-scribe-
log4j-0.20.2+320.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/hsqldb.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jackson-
core-asl-1.0.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jackson-mapper-asl-1.0.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jasper-
compiler-5.5.12.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jasper-
runtime-5.5.12.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jetty-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jetty-
util-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/junit-4.5.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/kfs-0.2.2.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/libfb303.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/libthrift.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/log4j-1.2.15.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/mockito-
all-1.8.2.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/mysql-connector-java-5.0.8-
bin.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/oro-2.0.8.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/servlet-
api-2.5-6.1.14.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/slf4j-
log4j12-1.4.3.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/xmlenc-0.52.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/hadoop-0.20.2/bin/../lib/jsp-2.1/jsp-
api-2.1.jar:/usr/local/hadoop/hadoop-0.20.2/lib/hadoop-lzo-0.4.4.jar org.apache.hadoop.mapred.TaskTracker
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

See,lzo library appeared in classpath,so questions are:
1,Here,lzo library appeared tow times,so if lzo library has been put in the $HADOOP_HOME/lib and native libraries have been put in
$HADOOP_HOME/lib/native,whether following settings  isn't needed:
hadoop-env.sh

export HADOOP_CLASSPATH=/usr/lib/hadoop/lib/hadoop-lzo-0.4.3.jar
export JAVA_LIBRARY_PATH=/usr/lib/hadoop/lib/native/Linux-i386-32 (or
the 64bit version)
2,the basic problem is that if lzo library appeared in the classpath,as above shows,whether this means that lzo library has been loaded
successfully,and if it was,why I get got error:"Caused by: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec" when run:

hadoop jar hadoop-examples-0.20.2+320.jar grep input output 'dfs[a-z.]+',
On Monday, August 16, 2010 06:50:28 am Josh Patterson wrote: