Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> InvalidProtocolBufferException while submitting crunch job to cluster


Copy link to this message
-
Re: InvalidProtocolBufferException while submitting crunch job to cluster
Thank you Shekhar, Harsh. I will follow and try to implement your
suggestions. I appreciate the help.
On Sat, Aug 31, 2013 at 11:12 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Your cluster is using HDFS HA, and therefore requires a little more
> configs than just fs.defaultFS/etc..
>
> You need to use the right set of cluster client configs. If you don't
> have them at /etc/hadoop/conf and /etc/hbase/conf on your cluster edge
> node to pull from, try asking your cluster administrator for a
> configuration set, and place their parent directories on your
> application's classpath.
>
> The first error deals with perhaps you also including a guava
> dependency in your project, which is different than the one
> transitively pulled in by hadoop-client via crunch. You should be able
> to use guava libs without needing an explicit dependency, and it would
> be the right needed version.
>
> The second error deals with your MR submission failing, cause the JT
> is using a staging directory over a HDFS HA, which uses a "logical"
> name of "bdatadev". A logical HA name needs other configs (typically
> in the hdfs-site.xml) that tell it which are the actual physical NNs
> under it - configs that you're missing here.
>
> On Sat, Aug 31, 2013 at 1:34 AM, Narlin M <[EMAIL PROTECTED]> wrote:
> > I am getting following exception while trying to submit a crunch pipeline
> > job to a remote hadoop cluster:
> >
> > Exception in thread "main" java.lang.RuntimeException: Cannot create job
> > output directory /tmp/crunch-324987940
> > at
> >
> org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.java:344)
> > at org.apache.crunch.impl.mr.MRPipeline.<init>(MRPipeline.java:125)
> > at test.CrunchTest.setup(CrunchTest.java:98)
> > at test.CrunchTest.main(CrunchTest.java:367)
> > Caused by: java.io.IOException: Failed on local exception:
> > com.google.protobuf.InvalidProtocolBufferException: Protocol message
> > end-group tag did not match expected tag.; Host Details : local host is:
> > "NARLIN/127.0.0.1"; destination host is: "<server_address>":50070;
> > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:759)
> > at org.apache.hadoop.ipc.Client.call(Client.java:1164)
> > at
> >
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
> > at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> >
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
> > at
> >
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
> > at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
> > at
> >
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:425)
> > at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1943)
> > at
> >
> org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:523)
> > at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1799)
> > at
> >
> org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.java:342)
> > ... 3 more
> > Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
> > message end-group tag did not match expected tag.
> > at
> >
> com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:73)
> > at
> >
> com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:124)
> > at
> >
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:213)
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:746)
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:238)