Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Configuring FLUME to use FailOverSinkProcessor...


+
Inder Pall 2012-07-12, 14:04
+
Jarek Jarcec Cecho 2012-07-12, 14:11
Copy link to this message
-
Re: Configuring FLUME to use FailOverSinkProcessor...
Hi Inder,
could you please advise what exactly was (or wasn't) happening to you? I've tried your configuration file on current trunk and it was working for me out of the box.

I was playing with it to see if your idea will work and I've ended up with attached configuration. It contains just few modification to yours:

* I've swapped priorities to firstly use the memory channel and fail over to file channel
* I've renamed target file prefixes to distinguish source of the events (memory or file channel)

Scenario that I've run:

* I turned on the flume agents, event's were correctly saved on HDFS
* I turned off name node, event's were not stored in file channel as expected

I'm not sure why the observed behaviour was different than expected and 'I'll investigate that later. Meantime, could you describe what exactly was happening to you?

Jarcec

On Thu, Jul 12, 2012 at 04:11:08PM +0200, Jarek Jarcec Cecho wrote:
> I'm just looking on that sir.
>
> Jarcec
>
> On Thu, Jul 12, 2012 at 07:34:31PM +0530, Inder Pall wrote:
> > Folks,
> >
> > for some reason updating the JIRA isn't trigerring an email....
> > I need feedback from FLUME DEVS on
> > FLUME-1045<https://issues.apache.org/jira/browse/FLUME-1045?focusedCommentId=13412737&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13412737>
> > wherein
> > i am trying to use failover sinkprocessor and a combination of file and
> > memory channel to achieve scribe like spooling/de-spooling.....facing some
> > issues here....
> >
> > --
> > Thanks,
> > - Inder
> >   Tech Platforms @Inmobi
> >   Linkedin - http://goo.gl/eR4Ub
+
Inder Pall 2012-07-13, 13:48
+
Inder Pall 2012-07-13, 13:51
+
Jarek Jarcec Cecho 2012-07-14, 06:05