Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Restarts without data loss


Copy link to this message
-
Re: Restarts without data loss
Hi,

On Sun, Jul 8, 2012 at 11:18 PM, Senthilvel Rangaswamy <[EMAIL PROTECTED]
> wrote:

> We are using Flume 1.2.0 with memory channel. When we rollout new
> configs/decorators
> we may need to restart flume at which point any events in memory channel
> is gone. Any
> ways to avoid this ?
>

One way to address this would be to make sure that the upstream sink or
client can be routed to a different agent when necessary. That way when you
do want to restart the file channel, you would first route all the traffic
elsewhere, drain the channel and then do the shutdown as necessary. Once
the system is back up, you could route the traffic back to this agent.

I am sure that there are multiple other ways of doing this.

Regards,
Arvind Prabhakar

>
> Thanks,
> --
> ..Senthil
>
> "If there's anything more important than my ego around, I want it
>  caught and shot now."
>                                                     - Douglas Adams.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB