Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> test lzo problem in hadoop


Copy link to this message
-
Re: test lzo problem in hadoop
i use yarn ,and i commented the following option and error is different
vi /etc/hadoop/conf/mapred-site.xml
<!--
        <property>
                <name>mapred.job.tracker</name>
                <value>CH22:8088</value>
        </property>
-->
#  hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-0.4.15.jar
com.hadoop.compression.lzo.DistributedLzoIndexer /alex/ttt.lzo
13/08/02 09:25:51 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/02 09:25:51 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]
13/08/02 09:25:52 INFO lzo.DistributedLzoIndexer: Adding LZO file
/alex/ttt.lzo to indexing list (no index currently exists)
13/08/02 09:25:52 WARN conf.Configuration: session.id is deprecated.
Instead, use dfs.metrics.session-id
13/08/02 09:25:52 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId13/08/02 09:25:52 WARN conf.Configuration: slave.host.name is deprecated.
Instead, use dfs.datanode.hostname
13/08/02 09:25:52 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
13/08/02 09:25:52 INFO input.FileInputFormat: Total input paths to process
: 1
13/08/02 09:25:53 INFO mapred.LocalJobRunner: OutputCommitter set in config
null
13/08/02 09:25:53 INFO mapred.JobClient: Running job:
job_local180628093_0001
13/08/02 09:25:53 INFO mapred.LocalJobRunner: OutputCommitter is
com.hadoop.mapreduce.LzoIndexOutputFormat$1
13/08/02 09:25:53 INFO mapred.LocalJobRunner: Waiting for map tasks
13/08/02 09:25:53 INFO mapred.LocalJobRunner: Starting task:
attempt_local180628093_0001_m_000000_0
13/08/02 09:25:53 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
13/08/02 09:25:53 INFO util.ProcessTree: setsid exited with exit code 0
13/08/02 09:25:53 INFO mapred.Task:  Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@338e18a3
13/08/02 09:25:53 INFO mapred.MapTask: Processing split:
hdfs://CH22:9000/alex/ttt.lzo:0+306
13/08/02 09:25:53 INFO mapred.LocalJobRunner: Map task executor complete.
13/08/02 09:25:53 WARN mapred.LocalJobRunner: job_local180628093_0001
java.lang.Exception: java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was
expected
        at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404)
Caused by: java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
        at
com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:47)
        at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:478)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:671)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
        at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:266)
        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
13/08/02 09:25:54 INFO mapred.JobClient:  map 0% reduce 0%
13/08/02 09:25:54 INFO mapred.JobClient: Job complete:
job_local180628093_0001
13/08/02 09:25:54 INFO mapred.JobClient: Counters: 0

On Mon, Jul 22, 2013 at 6:07 PM, Sandeep Nemuri <[EMAIL PROTECTED]>wrote:

> Try  this Ccommand
> hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.LzoIndexer /user/sample.txt.lzo
>
>
> On Mon, Jul 22, 2013 at 2:08 PM, ch huang <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB