Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Problem Events


+
Jeremy Karlson 2013-07-24, 21:52
+
Roshan Naik 2013-07-24, 22:36
Copy link to this message
-
Re: Problem Events
Or you could write a custom sink that removes this event (more work of course)
Thanks,
Hari
On Wednesday, July 24, 2013 at 3:36 PM, Roshan Naik wrote:

> if you have a way to identify such events.. you may be able to use the Regex interceptor to toss them out before they get into the channel.
>
>
>
> On Wed, Jul 24, 2013 at 2:52 PM, Jeremy Karlson <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > Hi everyone.  My Flume adventures continue.
> >
> > I'm in a situation now where I have a channel that's filling because a stubborn message is stuck.  The sink won't accept it (for whatever reason; I can go into detail but that's not my point here).  This just blocks up the channel entirely, because it goes back into the channel when the sink refuses.  Obviously, this isn't ideal.
> >
> > I'm wondering what mechanisms, if any, Flume has to deal with these situations.  Things that come to mind might be:
> >
> > 1. Ditch the event after n attempts.
> > 2. After n attempts, send the event to a "problem area" (maybe a different source / sink / channel?)  that someone can look at later.
> > 3. Some sort of mechanism that allows operators to manually kill these messages.
> >
> > I'm open to suggestions on alternatives as well.
> >
> > Thanks.
> >
> > -- Jeremy

+
Jeremy Karlson 2013-07-24, 22:56
+
Arvind Prabhakar 2013-07-25, 02:51
+
Jeremy Karlson 2013-07-25, 16:50
+
Arvind Prabhakar 2013-07-26, 00:35
+
Anat Rozenzon 2013-08-01, 07:59
+
Ashish 2013-08-01, 08:13
+
Anat Rozenzon 2013-08-01, 09:42
+
Jeremy Karlson 2013-08-01, 16:26
+
Roshan Naik 2013-08-01, 17:26
+
Paul Chavez 2013-08-01, 17:33
+
Arvind Prabhakar 2013-08-01, 22:25
+
Connor Woodson 2013-08-03, 01:27
+
Connor Woodson 2013-08-03, 06:56
+
Anat Rozenzon 2013-08-04, 05:45
+
Anat Rozenzon 2013-08-07, 05:40
+
Jonathan Cooper-Ellis 2013-08-07, 14:14
+
Anat Rozenzon 2013-08-08, 05:26
+
Connor Woodson 2013-08-10, 01:08
+
Hari Shreedharan 2013-08-07, 16:50
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB