Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Failover in Flume NG


Copy link to this message
-
Re: Failover in Flume NG
Roshan Naik 2012-10-16, 17:32
There is support for failover and load balancing. Take a look at sink
processors here
http://flume.apache.org/FlumeUserGuide.html#flume-sink-processors
You could have failover sinks configured to send it a different agent(i,e
collector).

-roshan
On Tue, Oct 16, 2012 at 10:09 AM, Pankaj Gupta <[EMAIL PROTECTED]>wrote:

> Hi,
>
>
> We are investigating using Flume for log aggregation in our organization
> and have this dilemma over going with Flume NG or OG. We really like the
> centralized configuration management in OG but that's not a must have. What
> we need most is reliability in face of machines going down (which is quite
> common on AWS). How is that handled in Flume NG. In OG, I believe, it was
> possible to define failover semantics so that messages could be routed to
> another collector if one collector went down. In Flume OG there is no
> concept of a special collector but collector is just an agent as I
> understand. I'm wondering how failover semantics are defined for these
> agents. It would be great if someone could elaborate on that.
>
> We also want to use a relatively new version of Hadoop e.g. 1.0.3 or
> higher. Is it possible to use Flume NG (or OG) with the new versions of
> Hadoop?
>
> Thanks in advance,
>
> Pankaj
>
> --
>
>
> *P* | (415) 677-9222 ext. 205 *F *| (415) 677-0895 | [EMAIL PROTECTED]
>
> Pankaj Gupta | Software Engineer
>
> *BrightRoll, Inc. *| Smart Video Advertising | www.brightroll.com
>
>
> United States | Canada | United Kingdom | Germany
>
>
> We're hiring<http://newton.newtonsoftware.com/career/CareerHome.action?clientId=8a42a12b3580e2060135837631485aa7>
> !
>
>