Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Re: Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing to HDFS sink


Copy link to this message
-
Re: Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing to HDFS sink
Glad to hear that was the issue.

dave
On Tue, Oct 29, 2013 at 12:53 AM, wesley dias <[EMAIL PROTECTED]>wrote:

> Thanks Roshan. But issue is now resolved. I compiled flume 1.4 from source
> using protobuf 2.5 and this solved the below mentioned issue. My guess is
> that the distribution available for flume 1.4 from the release site must
> have been compiled using protobuf 2.4
>
> Regards,
>
> Wesley
>
> ------------------------------
> Date: Mon, 28 Oct 2013 13:41:52 -0700
>
> Subject: Re: Exception in thread
> "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing to HDFS sink
> From: [EMAIL PROTECTED]
> To: [EMAIL PROTECTED]
>
> HDP already includes hadoop 2.2.0 + Flume 1.4. You'd be better of just
> using instead of building and testing that combo out on your own.
>
> http://hortonworks.com/products/hdp-2/?b=2#overview
>
>
> On Mon, Oct 28, 2013 at 1:19 PM, wesley dias <[EMAIL PROTECTED]>wrote:
>
> Hi David,
>
> Thanks for the quick reply.Does this mean that I would need to compile the
> flume from source using protobuf 2.5 instead of directly using the flume
> distribution available for download from the site? Is flume 1.4 based of
> protobuf 2.4?
> I am not sure of the same hence I am asking these questions.
>
> Regards,
>
> Wesley
> ------------------------------
> Date: Mon, 28 Oct 2013 16:11:16 -0400
> Subject: Re: Exception in thread
> "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing to HDFS sink
> From: [EMAIL PROTECTED]
> To: [EMAIL PROTECTED]
>
>
> Looks like you have a protocol buffer version problem. Do you have a
> dependency on 2.5.0 version of protocol buffers in your pom?
>
> see
>
> http://code.google.com/p/protobuf/issues/detail?id=493
>
>
>
> On Mon, Oct 28, 2013 at 4:02 PM, wesley dias <[EMAIL PROTECTED]>wrote:
>
> Hi,
>
> I am using the latest flume build 1.4 along with hadoop 2.2.0 to collect
> twitter stream and write to HDFS sink. However, on writing to HDFS I get
> the following error:
>
> 2013-10-28 09:41:23,481 (SinkRunner-PollingRunner-**DefaultSinkProcessor)
> [ERROR - org.apache.flume.sink.hdfs.**HDFSEventSink.process(**HDFSEventSink.java:422)]
> process failed
> java.lang.VerifyError: class org.apache.hadoop.hdfs.**protocol.proto.**
> ClientNamenodeProtocolProtos$**SetOwnerRequestProto overrides final
> method getUnknownFields.()Lcom/**google/protobuf/**UnknownFieldSet;
>     at java.lang.ClassLoader.**defineClass1(Native Method)
>     at java.lang.ClassLoader.**defineClass(ClassLoader.java:**800)
>     at java.security.**SecureClassLoader.defineClass(**
> SecureClassLoader.java:142)
>     at java.net.URLClassLoader.**defineClass(URLClassLoader.**java:449)
>     at java.net.URLClassLoader.**access$100(URLClassLoader.**java:71)
>     at java.net.URLClassLoader$1.run(**URLClassLoader.java:361)
>     at java.net.URLClassLoader$1.run(**URLClassLoader.java:355)
>     at java.security.**AccessController.doPrivileged(**Native Method)
>     at java.net.URLClassLoader.**findClass(URLClassLoader.java:**354)
>     at java.lang.ClassLoader.**loadClass(ClassLoader.java:**425)
>     at sun.misc.Launcher$**AppClassLoader.loadClass(**Launcher.java:308)
>     at java.lang.ClassLoader.**loadClass(ClassLoader.java:**358)
>     at java.lang.Class.**getDeclaredMethods0(Native Method)
>     at java.lang.Class.**privateGetDeclaredMethods(**Class.java:2531)
>     at java.lang.Class.**privateGetPublicMethods(Class.**java:2651)
>     at java.lang.Class.**privateGetPublicMethods(Class.**java:2661)
>     at java.lang.Class.getMethods(**Class.java:1467)
>
>  I am currently new to flume and any help shall be really appreciated.
>
> Regards,
>
> Wesley
>
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB