Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: test lzo problem in hadoop


Copy link to this message
-
Re: test lzo problem in hadoop
that's ok ,but why i can not use
com.hadoop.compression.lzo.DistributedLzoIndexer ?

# hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-0.4.15.jar
com.hadoop.compression.lzo.LzoIndexer  /alex/ttt.lzo
13/08/02 09:11:09 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/02 09:11:09 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]
13/08/02 09:11:10 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
/alex/ttt.lzo, size 0.00 GB...
13/08/02 09:11:10 WARN conf.Configuration: hadoop.native.lib is deprecated.
Instead, use io.native.lib.available
13/08/02 09:11:10 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.19
seconds (0.00 MB/s).  Index size is 0.01 KB.
On Mon, Jul 22, 2013 at 6:07 PM, Sandeep Nemuri <[EMAIL PROTECTED]>wrote:

> Try  this Ccommand
> hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.LzoIndexer /user/sample.txt.lzo
>
>
> On Mon, Jul 22, 2013 at 2:08 PM, ch huang <[EMAIL PROTECTED]> wrote:
>
>> anyone can help?
>>
>> # sudo -u hdfs hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-0.4.15.jar
>> com.hadoop.compression.lzo.DistributedLzoIndexer
>> /alex/test_lzo/sqoop-1.99.2-bin-hadoop200.tar.gz.lzo
>> 13/07/22 16:33:50 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/07/22 16:33:50 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]
>> 13/07/22 16:33:50 INFO lzo.DistributedLzoIndexer: Adding LZO file
>> /alex/test_lzo/sqoop-1.99.2-bin-hadoop200.tar.gz.lzo to indexing list (no
>> index currently exists)
>> 13/07/22 16:33:50 ERROR security.UserGroupInformation:
>> PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException:
>> Failed on local exception:
>> com.google.protobuf.InvalidProtocolBufferException: Protocol message
>> end-group tag did not match expected tag.; Host Details : local host is:
>> "CH22/192.168.10.22"; destination host is: "CH22":8088;
>> Exception in thread "main" java.io.IOException: Failed on local
>> exception: com.google.protobuf.InvalidProtocolBufferException: Protocol
>> message end-group tag did not match expected tag.; Host Details : local
>> host is: "CH22/192.168.10.22"; destination host is: "CH22":8088;
>>         at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:763)
>>         at org.apache.hadoop.ipc.Client.call(Client.java:1229)
>>         at
>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
>>         at org.apache.hadoop.mapred.$Proxy10.getStagingAreaDir(Unknown
>> Source)
>>         at
>> org.apache.hadoop.mapred.JobClient.getStagingAreaDir(JobClient.java:1324)
>>         at
>> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:102)
>>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:951)
>>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>>         at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945)
>>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
>>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596)
>>         at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:111)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>         at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB