Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - FileChannel - WARN on restart


Copy link to this message
-
Re: FileChannel - WARN on restart
Brock Noland 2013-01-19, 06:37
Yeah that should be info. Just means we'll be reading a little more data than needed but still waiting to process data until the correct write order id.  

Long term I'd like see a new rev of the file format which includes prefixed checksums and sync records for random reads (for skipping to the correct location based in write order id) included.  

--
Brock Noland
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Friday, January 18, 2013 at 10:26 PM, Connor Woodson wrote:

> When I restart a flume agent with a file channel (ctrl^c to stop the
> process), I get the following warning when the channel is empty (not sure
> what happens otherwise):
>
> Checkpoint for file(<path>) is: 1358539648673, which is beyond the
> requested checkpoint time: 1358539648673 and position 0
>
> Looking at the code for that warning
> (link<https://github.com/apache/flume/blob/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java#L462>),
> it looks like it always appears when the file channel is empty (rather,
> when the checkpoint position is the beginning of the file). Is that
> necessary?
>
> - Connor