Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: test lzo problem in hadoop


Copy link to this message
-
Re: test lzo problem in hadoop
that's ok ,but why i can not use
com.hadoop.compression.lzo.DistributedLzoIndexer ?

# hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-0.4.15.jar
com.hadoop.compression.lzo.LzoIndexer  /alex/ttt.lzo
13/08/02 09:11:09 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/02 09:11:09 INFO lzo.LzoCodec: Successfully loaded & initialized
native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]
13/08/02 09:11:10 INFO lzo.LzoIndexer: [INDEX] LZO Indexing file
/alex/ttt.lzo, size 0.00 GB...
13/08/02 09:11:10 WARN conf.Configuration: hadoop.native.lib is deprecated.
Instead, use io.native.lib.available
13/08/02 09:11:10 INFO lzo.LzoIndexer: Completed LZO Indexing in 0.19
seconds (0.00 MB/s).  Index size is 0.01 KB.
On Mon, Jul 22, 2013 at 6:07 PM, Sandeep Nemuri <[EMAIL PROTECTED]>wrote:

> Try  this Ccommand
> hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar
> com.hadoop.compression.lzo.LzoIndexer /user/sample.txt.lzo
>
>
> On Mon, Jul 22, 2013 at 2:08 PM, ch huang <[EMAIL PROTECTED]> wrote:
>
>> anyone can help?
>>
>> # sudo -u hdfs hadoop jar /usr/lib/hadoop/lib/hadoop-lzo-0.4.15.jar
>> com.hadoop.compression.lzo.DistributedLzoIndexer
>> /alex/test_lzo/sqoop-1.99.2-bin-hadoop200.tar.gz.lzo
>> 13/07/22 16:33:50 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
>> 13/07/22 16:33:50 INFO lzo.LzoCodec: Successfully loaded & initialized
>> native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]
>> 13/07/22 16:33:50 INFO lzo.DistributedLzoIndexer: Adding LZO file
>> /alex/test_lzo/sqoop-1.99.2-bin-hadoop200.tar.gz.lzo to indexing list (no
>> index currently exists)
>> 13/07/22 16:33:50 ERROR security.UserGroupInformation:
>> PriviledgedActionException as:hdfs (auth:SIMPLE) cause:java.io.IOException:
>> Failed on local exception:
>> com.google.protobuf.InvalidProtocolBufferException: Protocol message
>> end-group tag did not match expected tag.; Host Details : local host is:
>> "CH22/192.168.10.22"; destination host is: "CH22":8088;
>> Exception in thread "main" java.io.IOException: Failed on local
>> exception: com.google.protobuf.InvalidProtocolBufferException: Protocol
>> message end-group tag did not match expected tag.; Host Details : local
>> host is: "CH22/192.168.10.22"; destination host is: "CH22":8088;
>>         at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:763)
>>         at org.apache.hadoop.ipc.Client.call(Client.java:1229)
>>         at
>> org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225)
>>         at org.apache.hadoop.mapred.$Proxy10.getStagingAreaDir(Unknown
>> Source)
>>         at
>> org.apache.hadoop.mapred.JobClient.getStagingAreaDir(JobClient.java:1324)
>>         at
>> org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:102)
>>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:951)
>>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>>         at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945)
>>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
>>         at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596)
>>         at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.run(DistributedLzoIndexer.java:111)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>         at
>> com.hadoop.compression.lzo.DistributedLzoIndexer.main(DistributedLzoIndexer.java:115)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>