Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> InvalidProtocolBufferException while submitting crunch job to cluster


Copy link to this message
-
InvalidProtocolBufferException while submitting crunch job to cluster
I am getting following exception while trying to submit a crunch pipeline
job to a remote hadoop cluster:

Exception in thread "main" java.lang.RuntimeException: Cannot create job
output directory /tmp/crunch-324987940
at
org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.java:344)
at org.apache.crunch.impl.mr.MRPipeline.<init>(MRPipeline.java:125)
at test.CrunchTest.setup(CrunchTest.java:98)
at test.CrunchTest.main(CrunchTest.java:367)
Caused by: java.io.IOException: Failed on local exception:
com.google.protobuf.InvalidProtocolBufferException: Protocol message
end-group tag did not match expected tag.; Host Details : local host is:
"NARLIN/127.0.0.1"; destination host is: "<server_address>":50070;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:759)
at org.apache.hadoop.ipc.Client.call(Client.java:1164)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:425)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1943)
at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:523)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1799)
at
org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.java:342)
... 3 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
message end-group tag did not match expected tag.
at
com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:73)
at
com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:124)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:213)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:746)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:238)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:282)
at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:760)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:288)
at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:752)
at
org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:985)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:882)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:813)
0    [Thread-3] WARN  org.apache.hadoop.util.ShutdownHookManager  -
ShutdownHook 'ClientFinalizer' failed, java.lang.NoSuchMethodError:
com.google.common.collect.LinkedListMultimap.values()Ljava/util/List;
java.lang.NoSuchMethodError:
com.google.common.collect.LinkedListMultimap.values()Ljava/util/List;
at org.apache.hadoop.hdfs.SocketCache.clear(SocketCache.java:135)
at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:672)
at
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:539)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2308)
at
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2324)
at
org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)

Google search on this error yielded solutions that asked to confirm that
/etc/hosts file contained the entry for NARLIN which it does in my case.

Here's the code that I am using to set up the MRPipeline:

Configuration conf = HBaseConfiguration.create();

conf.set("fs.defaultFS", "hdfs://<server_address>:50070");
conf.set("mapred.job.tracker", "<server_address>:50030");

System.out.println("Hadoop configuration created.");
System.out.println("Initializing crunch pipeline ...");

conf.set("mapred.jar", "<path_to_jar_file>");

pipeline = new MRPipeline(getClass(), "crunchjobtest", conf);

Has anyone faced this issue before and knows how to resolve it/point out if
I am missing anything?

Thanks for the help.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB