Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - Configuring FLUME to use FailOverSinkProcessor...


Copy link to this message
-
Re: Configuring FLUME to use FailOverSinkProcessor...
Inder Pall 2012-07-13, 13:48
Jarek,

thanks for taking out time to try this..yeah i meant mem channel to be used
first and then file channel for some reason i thought lower number means
higher priority pardon the ignorance of not looking at the
documentation.....

i have a dated code-base of trunk around 10-12 days...the agent comes up
fine but the sequence generator source wasn't sending any events...or
actually i didn't see anything in the logs....

Thanks,
 - Inder

On Fri, Jul 13, 2012 at 3:30 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:

> Hi Inder,
> could you please advise what exactly was (or wasn't) happening to you?
> I've tried your configuration file on current trunk and it was working for
> me out of the box.
>
> I was playing with it to see if your idea will work and I've ended up with
> attached configuration. It contains just few modification to yours:
>
> * I've swapped priorities to firstly use the memory channel and fail over
> to file channel
> * I've renamed target file prefixes to distinguish source of the events
> (memory or file channel)
>
> Scenario that I've run:
>
> * I turned on the flume agents, event's were correctly saved on HDFS
> * I turned off name node, event's were not stored in file channel as
> expected
>
> I'm not sure why the observed behaviour was different than expected and
> 'I'll investigate that later. Meantime, could you describe what exactly was
> happening to you?
>
> Jarcec
>
> On Thu, Jul 12, 2012 at 04:11:08PM +0200, Jarek Jarcec Cecho wrote:
> > I'm just looking on that sir.
> >
> > Jarcec
> >
> > On Thu, Jul 12, 2012 at 07:34:31PM +0530, Inder Pall wrote:
> > > Folks,
> > >
> > > for some reason updating the JIRA isn't trigerring an email....
> > > I need feedback from FLUME DEVS on
> > > FLUME-1045<
> https://issues.apache.org/jira/browse/FLUME-1045?focusedCommentId=13412737&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13412737
> >
> > > wherein
> > > i am trying to use failover sinkprocessor and a combination of file and
> > > memory channel to achieve scribe like spooling/de-spooling.....facing
> some
> > > issues here....
> > >
> > > --
> > > Thanks,
> > > - Inder
> > >   Tech Platforms @Inmobi
> > >   Linkedin - http://goo.gl/eR4Ub
>
>
>
--
Thanks,
- Inder
  Tech Platforms @Inmobi
  Linkedin - http://goo.gl/eR4Ub