Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> log file flush?


Copy link to this message
-
Re: log file flush?
Jason,

Although messages are always written to the log segment file, they
initially are only in the file system's pagecache. As Swapnil mentioned
earlier, messages are flushed to disk periodically. If you do a clean
shutdown (kill -15), we close all log file, which should flush all dirty
data to disk. If you do a hard kill or your machine just crashed, the
unflushed data may be lost. The data that you saw in the .log file can be
just in the pagecache.

Thanks,

Jun

On Tue, Feb 19, 2013 at 4:05 AM, Jason Huang <[EMAIL PROTECTED]> wrote:

> Thanks for response.
>
> My confusion is that - once I see the message content in the .log
> file, doesn't that mean the message has already been flushed to the
> hard drive? Why would those messages still get lost if someone
> manually kill the process (or if the server crashes unexpectedly)?
>
> Jason
>
> On Tue, Feb 19, 2013 at 6:53 AM, Swapnil Ghike <[EMAIL PROTECTED]>
> wrote:
> > Correction - The flush happens based on *number of messages* and time
> > limits, whichever is hit first.
> >
> >
> >
> > On 2/19/13 3:50 AM, "Swapnil Ghike" <[EMAIL PROTECTED]> wrote:
> >
> >>The flush happens based on size and time limits,
> >>whichever is hit first.
> >
>