Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> FileChannel - WARN on restart


Copy link to this message
-
Re: FileChannel - WARN on restart
Yeah that should be info. Just means we'll be reading a little more data than needed but still waiting to process data until the correct write order id.  

Long term I'd like see a new rev of the file format which includes prefixed checksums and sync records for random reads (for skipping to the correct location based in write order id) included.  

--
Brock Noland
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Friday, January 18, 2013 at 10:26 PM, Connor Woodson wrote:

> When I restart a flume agent with a file channel (ctrl^c to stop the
> process), I get the following warning when the channel is empty (not sure
> what happens otherwise):
>
> Checkpoint for file(<path>) is: 1358539648673, which is beyond the
> requested checkpoint time: 1358539648673 and position 0
>
> Looking at the code for that warning
> (link<https://github.com/apache/flume/blob/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java#L462>),
> it looks like it always appears when the file channel is empty (rather,
> when the checkpoint position is the beginning of the file). Is that
> necessary?
>
> - Connor

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB