-Re: where does the in-memory log to file persistence take place
S Ahmed 2012-05-10, 04:35
couldn't this be written in a way that it doesn't block? or limits the
time, like it makes a copy of the reference, then replaces it when a newed
up channel (synchronized).
Oh, this isn't possible because the var is mapped to a file at the o/s
On Tue, May 8, 2012 at 5:38 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> filechannel.force() always fully syncs the file to disk. This is done
> irrespective of message boundaries. The file is locked during this
> time so other appends are blocked.
> On Tue, May 8, 2012 at 1:44 PM, S Ahmed <[EMAIL PROTECTED]> wrote:
> > Slowly trying to understand it, have to wramp up on my scala.
> > When the flush/sink occurrs, does it pull items of the collection 1 by 1
> > does it do this in bulk somehow while locking the collection?
> > On Mon, May 7, 2012 at 3:14 PM, Neha Narkhede <[EMAIL PROTECTED]
> >> Ahmed,
> >> The related code is in kafka.log.*. The message to file persistence is
> >> inside FileMessageSet.scala.
> >> Thanks,
> >> Neha
> >> On Mon, May 7, 2012 at 12:12 PM, S Ahmed <[EMAIL PROTECTED]> wrote:
> >> > I can barely read scala, but I'm curious where the applications
> >> > the operation of taking the in-memory log and persisting it to the
> >> > database, all the while accepting new log messages and removing the
> >> of
> >> > the messages that have been persisted to disk.
> >> >
> >> > I'm guessing you have used the concurrenthashmap where the key is a
> >> topic,
> >> > and once the flush timeout has been reached a background thread will
> >> > somehow persist and remove the keys.
> >> >