Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> InvalidProtocolBufferException while submitting crunch job to cluster


Copy link to this message
-
Re: InvalidProtocolBufferException while submitting crunch job to cluster
Thank you Shekhar, Harsh. I will follow and try to implement your
suggestions. I appreciate the help.
On Sat, Aug 31, 2013 at 11:12 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Your cluster is using HDFS HA, and therefore requires a little more
> configs than just fs.defaultFS/etc..
>
> You need to use the right set of cluster client configs. If you don't
> have them at /etc/hadoop/conf and /etc/hbase/conf on your cluster edge
> node to pull from, try asking your cluster administrator for a
> configuration set, and place their parent directories on your
> application's classpath.
>
> The first error deals with perhaps you also including a guava
> dependency in your project, which is different than the one
> transitively pulled in by hadoop-client via crunch. You should be able
> to use guava libs without needing an explicit dependency, and it would
> be the right needed version.
>
> The second error deals with your MR submission failing, cause the JT
> is using a staging directory over a HDFS HA, which uses a "logical"
> name of "bdatadev". A logical HA name needs other configs (typically
> in the hdfs-site.xml) that tell it which are the actual physical NNs
> under it - configs that you're missing here.
>
> On Sat, Aug 31, 2013 at 1:34 AM, Narlin M <[EMAIL PROTECTED]> wrote:
> > I am getting following exception while trying to submit a crunch pipeline
> > job to a remote hadoop cluster:
> >
> > Exception in thread "main" java.lang.RuntimeException: Cannot create job
> > output directory /tmp/crunch-324987940
> > at
> >
> org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.java:344)
> > at org.apache.crunch.impl.mr.MRPipeline.<init>(MRPipeline.java:125)
> > at test.CrunchTest.setup(CrunchTest.java:98)
> > at test.CrunchTest.main(CrunchTest.java:367)
> > Caused by: java.io.IOException: Failed on local exception:
> > com.google.protobuf.InvalidProtocolBufferException: Protocol message
> > end-group tag did not match expected tag.; Host Details : local host is:
> > "NARLIN/127.0.0.1"; destination host is: "<server_address>":50070;
> > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:759)
> > at org.apache.hadoop.ipc.Client.call(Client.java:1164)
> > at
> >
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
> > at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> >
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
> > at
> >
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
> > at com.sun.proxy.$Proxy11.mkdirs(Unknown Source)
> > at
> >
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:425)
> > at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1943)
> > at
> >
> org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:523)
> > at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1799)
> > at
> >
> org.apache.crunch.impl.mr.MRPipeline.createTempDirectory(MRPipeline.java:342)
> > ... 3 more
> > Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
> > message end-group tag did not match expected tag.
> > at
> >
> com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:73)
> > at
> >
> com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:124)
> > at
> >
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:213)
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:746)
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:238)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB