Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - issue about try lzo in hadoop


Copy link to this message
-
issue about try lzo in hadoop
ch huang 2013-08-06, 06:39
i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run test
,it failed,why?

# sudo -u hdfs hadoop jar
/usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
com.hadoop.compression.lzo.LzoIndexer /test
13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
13/08/06 22:28:04 INFO lzo.LzoIndexer:   [INDEX] LZO Indexing file
hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is deprecated.
Instead, use io.native.lib.available
13/08/06 22:28:04 INFO lzo.LzoIndexer:   Completed LZO Indexing in 0.18
seconds (0.05 MB/s).  Index size is 0.01 KB.
[root@node1 ~]# echo $?
0
]# sudo -u hdfs hadoop jar
/usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
com.hadoop.compression.lzo.DistributedLzoIndexer /test
13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
exists)
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/Job
        at
com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at
com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.Job
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        ... 9 more
+
Sandeep Nemuri 2013-08-06, 07:27