Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> issue about try lzo in hadoop


Copy link to this message
-
issue about try lzo in hadoop
i use yarn ,and hadoop version is CDH4.3, lzo installed,but when i run test
,it failed,why?

# sudo -u hdfs hadoop jar
/usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
com.hadoop.compression.lzo.LzoIndexer /test
13/08/06 22:28:03 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/06 22:28:03 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
13/08/06 22:28:04 INFO lzo.LzoIndexer: LZO Indexing directory /test...
13/08/06 22:28:04 INFO lzo.LzoIndexer:   [INDEX] LZO Indexing file
hdfs://mycluster/test/install.log.lzo, size 0.00 GB...
13/08/06 22:28:04 WARN conf.Configuration: hadoop.native.lib is deprecated.
Instead, use io.native.lib.available
13/08/06 22:28:04 INFO lzo.LzoIndexer:   Completed LZO Indexing in 0.18
seconds (0.05 MB/s).  Index size is 0.01 KB.
[root@node1 ~]# echo $?
0
]# sudo -u hdfs hadoop jar
/usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
com.hadoop.compression.lzo.DistributedLzoIndexer /test
13/08/06 22:30:34 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/06 22:30:34 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
13/08/06 22:30:35 INFO lzo.DistributedLzoIndexer: Adding LZO file
hdfs://mycluster/test/install.log.lzo to indexing list (no index currently
exists)
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/Job
        at
com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:90)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at
com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.Job
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
        ... 9 more
+
Sandeep Nemuri 2013-08-06, 07:27
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB