Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Failover in Flume NG


Copy link to this message
-
Re: Failover in Flume NG
There is support for failover and load balancing. Take a look at sink
processors here
http://flume.apache.org/FlumeUserGuide.html#flume-sink-processors
You could have failover sinks configured to send it a different agent(i,e
collector).

-roshan
On Tue, Oct 16, 2012 at 10:09 AM, Pankaj Gupta <[EMAIL PROTECTED]>wrote:

> Hi,
>
>
> We are investigating using Flume for log aggregation in our organization
> and have this dilemma over going with Flume NG or OG. We really like the
> centralized configuration management in OG but that's not a must have. What
> we need most is reliability in face of machines going down (which is quite
> common on AWS). How is that handled in Flume NG. In OG, I believe, it was
> possible to define failover semantics so that messages could be routed to
> another collector if one collector went down. In Flume OG there is no
> concept of a special collector but collector is just an agent as I
> understand. I'm wondering how failover semantics are defined for these
> agents. It would be great if someone could elaborate on that.
>
> We also want to use a relatively new version of Hadoop e.g. 1.0.3 or
> higher. Is it possible to use Flume NG (or OG) with the new versions of
> Hadoop?
>
> Thanks in advance,
>
> Pankaj
>
> --
>
>
> *P* | (415) 677-9222 ext. 205 *F *| (415) 677-0895 | [EMAIL PROTECTED]
>
> Pankaj Gupta | Software Engineer
>
> *BrightRoll, Inc. *| Smart Video Advertising | www.brightroll.com
>
>
> United States | Canada | United Kingdom | Germany
>
>
> We're hiring<http://newton.newtonsoftware.com/career/CareerHome.action?clientId=8a42a12b3580e2060135837631485aa7>
> !
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB