Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> test lzo problem in hadoop


Copy link to this message
-
Re: test lzo problem in hadoop
i use yarn ,and i commented the following option and error is different
vi /etc/hadoop/conf/mapred-site.xml
<!--
        <property>
                <name>mapred.job.tracker</name>
                <value>CH22:8088</value>
        </property>
-->
#  hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-0.4.15.jar
com.hadoop.compression.lzo.DistributedLzoIndexer /alex/ttt.lzo
13/08/02 09:25:51 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/02 09:25:51 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]
13/08/02 09:25:52 INFO lzo.DistributedLzoIndexer: Adding LZO file
/alex/ttt.lzo to indexing list (no index currently exists)
13/08/02 09:25:52 WARN conf.Configuration: session.id is deprecated.
Instead, use dfs.metrics.session-id
13/08/02 09:25:52 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId13/08/02 09:25:52 WARN conf.Configuration: slave.host.name is deprecated.
Instead, use dfs.datanode.hostname
13/08/02 09:25:52 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
13/08/02 09:25:52 INFO input.FileInputFormat: Total input paths to process
: 1
13/08/02 09:25:53 INFO mapred.LocalJobRunner: OutputCommitter set in config
null
13/08/02 09:25:53 INFO mapred.JobClient: Running job:
job_local180628093_0001
13/08/02 09:25:53 INFO mapred.LocalJobRunner: OutputCommitter is
com.hadoop.mapreduce.LzoIndexOutputFormat$1
13/08/02 09:25:53 INFO mapred.LocalJobRunner: Waiting for map tasks
13/08/02 09:25:53 INFO mapred.LocalJobRunner: Starting task:
attempt_local180628093_0001_m_000000_0
13/08/02 09:25:53 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
13/08/02 09:25:53 INFO util.ProcessTree: setsid exited with exit code 0
13/08/02 09:25:53 INFO mapred.Task:  Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@338e18a3
13/08/02 09:25:53 INFO mapred.MapTask: Processing split:
hdfs://CH22:9000/alex/ttt.lzo:0+306
13/08/02 09:25:53 INFO mapred.LocalJobRunner: Map task executor complete.
13/08/02 09:25:53 WARN mapred.LocalJobRunner: job_local180628093_0001
java.lang.Exception: java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was
expected
        at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404)
Caused by: java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
        at
com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:47)
        at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:478)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:671)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
        at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:266)
        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
13/08/02 09:25:54 INFO mapred.JobClient:  map 0% reduce 0%
13/08/02 09:25:54 INFO mapred.JobClient: Job complete:
job_local180628093_0001
13/08/02 09:25:54 INFO mapred.JobClient: Counters: 0

On Mon, Jul 22, 2013 at 6:07 PM, Sandeep Nemuri <[EMAIL PROTECTED]>wrote:

> Try  this Ccommand
> hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.LzoIndexer /user/sample.txt.lzo
>
>
> On Mon, Jul 22, 2013 at 2:08 PM, ch huang <[EMAIL PROTECTED]> wrote: