Inder Pall 2012-07-12, 14:04
Jarek Jarcec Cecho 2012-07-12, 14:11
Jarek Jarcec Cecho 2012-07-13, 10:00
Inder Pall 2012-07-13, 13:48
-Re: Configuring FLUME to use FailOverSinkProcessor...
Inder Pall 2012-07-13, 13:51
So if this thing works what are people's thoughts about using it for PROD
Jarek, the reason i had hdfs data sink locations same for both HDFS sinks
was to ensure that the spooled data also makes to the final location...so
the test to try out would be to check all numbers generated by the seqeunce
generator source are available once the tests are finished...
btw, the avro-sink & avro source combination for file channel is a little
heavy as it's local..perhaps something lighter could be better.
* I turned off name node, event's were not stored in file channel as
>> here you meant they were indeed saved into file channel and drained from
memory channel right??
On Fri, Jul 13, 2012 at 7:18 PM, Inder Pall <[EMAIL PROTECTED]> wrote:
> thanks for taking out time to try this..yeah i meant mem channel to be
> used first and then file channel for some reason i thought lower number
> means higher priority pardon the ignorance of not looking at the
> i have a dated code-base of trunk around 10-12 days...the agent comes up
> fine but the sequence generator source wasn't sending any events...or
> actually i didn't see anything in the logs....
> - Inder
> On Fri, Jul 13, 2012 at 3:30 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>> Hi Inder,
>> could you please advise what exactly was (or wasn't) happening to you?
>> I've tried your configuration file on current trunk and it was working for
>> me out of the box.
>> I was playing with it to see if your idea will work and I've ended up
>> with attached configuration. It contains just few modification to yours:
>> * I've swapped priorities to firstly use the memory channel and fail over
>> to file channel
>> * I've renamed target file prefixes to distinguish source of the events
>> (memory or file channel)
>> Scenario that I've run:
>> * I turned on the flume agents, event's were correctly saved on HDFS
>> * I turned off name node, event's were not stored in file channel as
>> I'm not sure why the observed behaviour was different than expected and
>> 'I'll investigate that later. Meantime, could you describe what exactly was
>> happening to you?
>> On Thu, Jul 12, 2012 at 04:11:08PM +0200, Jarek Jarcec Cecho wrote:
>> > I'm just looking on that sir.
>> > Jarcec
>> > On Thu, Jul 12, 2012 at 07:34:31PM +0530, Inder Pall wrote:
>> > > Folks,
>> > >
>> > > for some reason updating the JIRA isn't trigerring an email....
>> > > I need feedback from FLUME DEVS on
>> > > FLUME-1045<
>> > > wherein
>> > > i am trying to use failover sinkprocessor and a combination of file
>> > > memory channel to achieve scribe like spooling/de-spooling.....facing
>> > > issues here....
>> > >
>> > > --
>> > > Thanks,
>> > > - Inder
>> > > Tech Platforms @Inmobi
>> > > Linkedin - http://goo.gl/eR4Ub
> - Inder
> Tech Platforms @Inmobi
> Linkedin - http://goo.gl/eR4Ub
Tech Platforms @Inmobi
Linkedin - http://goo.gl/eR4Ub
Jarek Jarcec Cecho 2012-07-14, 06:05