Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> flume.EventDeliveryException: Failed to send events


+
Denis Lowe 2013-02-06, 06:23
+
Juhani Connolly 2013-02-06, 08:21
+
Denis Lowe 2013-02-06, 18:10
+
Chris Neal 2013-04-16, 18:38
Copy link to this message
-
Re: flume.EventDeliveryException: Failed to send events
Looks like you are hitting Avro IPC timeouts - you should probably increase it, especially if you are talking over WAN.

--
Hari Shreedharan
On Tuesday, April 16, 2013 at 11:38 AM, Chris Neal wrote:

> I'm seeing the same thing :)
>
> Mine is all on a local LAN though, so the fact that an RPC call doesn't reply in 10000ms or 20000ms is quite odd.  My configuration is for the most part the same as Denis' configuration.  Two tiered system, ExecSources running tail -F on log files to an AvroSink, to an AvroSource, loading into HDFS on the back tier.
>
> I, too, see this on the AvroSink
>
> Either (A):
> [2013-04-15 23:57:14.827] [org.apache.flume.sink.LoadBalancingSinkProcessor] [ WARN] [SinkRunner-PollingRunner-LoadBalancingSinkProcessor] []  (LoadBalancingSinkProcessor.java:process:154) Sink failed to consume event. Attempting next sink if available.
> org.apache.flume.EventDeliveryException: Failed to send events
>         at org.apache.flume.sink.AvroSink.process(AvroSink.java:324)
>         at org.apache.flume.sink.LoadBalancingSinkProcessor.process(LoadBalancingSinkProcessor.java:151)
>         at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoopjt01.pegs.com (http://hadoopjt01.pegs.com), port: 10000 }: Failed to send batch
>         at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:236)
>         at org.apache.flume.sink.AvroSink.process(AvroSink.java:308)
>         ... 3 more
> Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoopjt01.pegs.com (http://hadoopjt01.pegs.com), port: 10000 }: Handshake timed out after 20000ms
>         at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:280)
>         at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:224)
>         ... 4 more
> Caused by: java.util.concurrent.TimeoutException
>         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:228)
>         at java.util.concurrent.FutureTask.get(FutureTask.java:91)
>         at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:278)
>         ... 5 more
>
> or (B):
> [2013-04-15 19:49:01.135] [org.apache.flume.sink.LoadBalancingSinkProcessor] [ WARN] [SinkRunner-PollingRunner-LoadBalancingSinkProcessor] []  (LoadBalancingSinkProcessor.java:process:154) Sink failed to consume event. Attempting next sink if available.
> org.apache.flume.EventDeliveryException: Failed to send events
>         at org.apache.flume.sink.AvroSink.process(AvroSink.java:324)
>         at org.apache.flume.sink.LoadBalancingSinkProcessor.process(LoadBalancingSinkProcessor.java:151)
>         at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoopjt01.pegs.com (http://hadoopjt01.pegs.com), port: 10000 }: Failed to send batch
>         at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:236)
>         at org.apache.flume.sink.AvroSink.process(AvroSink.java:308)
>         ... 3 more
> Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: hadoopjt01.pegs.com (http://hadoopjt01.pegs.com), port: 10000 }: RPC request timed out
>         at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:321)
>         at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:295)
>         at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:224)
>         ... 4 more
> Caused by: java.util.concurrent.TimeoutException
>         at org.apache.avro.ipc.CallFuture.get(CallFuture.java:132)
>         at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:310)
>         ... 6 more
+
Chris Neal 2013-04-16, 19:07
+
Hari Shreedharan 2013-04-16, 19:19
+
Brock Noland 2013-04-16, 19:26
+
Chris Neal 2013-04-16, 19:49
+
Brock Noland 2013-04-16, 19:57
+
Chris Neal 2013-04-16, 20:05
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB