Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> flume write ahead log is finished


+
Shengjie Min 2012-12-04, 17:26
Copy link to this message
-
Re: flume write ahead log is finished
Generally you should be fine to bring down the file channel at any
point. That is why we have the WAL.

If you wanted to be sure the channel was empty, we do export the
channel size (depth).

On Tue, Dec 4, 2012 at 11:26 AM, Shengjie Min <[EMAIL PROTECTED]> wrote:
> Hi,
>
> When it comes to maintenance outage, flume needs to be shutdown and offline
> for a while. We want to make sure flume's WAL is completed before we shut
> down the process. Is there anyway or any metrics exposed can indicate WAL is
> fully processed so that it's safe to bring it down gracefully?
>
> Of coz, the assumption here is that, the source of flume is not writing any
> new data, flume only has a certain size backlog needs to be cleaned up.
>
>
> --
> All the best,
> Shengjie Min
>

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Shengjie Min 2012-12-04, 21:02
+
Brock Noland 2012-12-04, 21:08
+
Shengjie Min 2012-12-04, 21:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB