-Re: Proper documentation for setting up sink groups
Bhaskar V. Karambelkar 2012-08-23, 18:43
Yes I did read the whole guide end to end.
But I still have doubts
The fact that multiple sinks can feed from the same channel is news to me.
I don't see it explicitly mentioned in the docs,
so i guess I assumed wrongly, that only one sink can feed from a channel.
a)Can you explain in detail , how having multiple sinks taking events from
one channel, is useful in a "fast source slow sinks" scenario ?
b) Also if I read your explanation below correctly there are 3 possible
1) multiple sinks feeding from a single channel , with the default sink
processor this will be like a multiplexing channel with all sinks getting
all the events that come in the channel.
2) multiple sinks feeding from a single channel , with fail_over sink
processor, only one sink will get the events at a give time, with flume
failing over to next available sink in case the first one fails ?
3) multiple sinks feeding from a single channel, with
load balancing processor, with all sinks getting events in a
Is this a correct assumption ? I am aware of #2 and #3, not sure about #1.
On Thu, Aug 23, 2012 at 12:43 PM, Hari Shreedharan <
[EMAIL PROTECTED]> wrote:
> Did you read this:
> That explains how to use sink groups. Also there is nothing wrong with
> multiple sinks taking events from one channel. This is an especially useful
> configuration if you have a very fast source and much slower sinks.
> Hari Shreedharan
> On Thursday, August 23, 2012 at 9:28 AM, Bhaskar V. Karambelkar wrote:
> > The sink group document doesn't mention anything about how
> > to hook up sink groups to the rest of the config in order to work.
> > e.g. under normal circumstances one channel is linked with one sink.
> > But for failover sink group , looks like both the sinks should be hooked
> up to the same channel,
> > but this is not mentioned any where.
> > Similarly, what exactly needs to be done for load balancing sink ?
> > thanks