Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - Time delay question


Copy link to this message
-
Re: Time delay question
John Vines 2012-08-08, 19:05
Once BatchWriter's return from flush() or when they automatically do a
flush in the background (when their buffers get sufficiently large), the
data is availble for scanners. At that point the data is at least in the in
memory map, as well as the walog.

As the in memory maps for the tablets for the table fill, they will write
to disk. And when there are more files of given sizes than desired, major
compaction will occur which will compact files to a file.

So when you scan, data is merged from the in memory map and the files.

John

On Wed, Aug 8, 2012 at 2:09 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> Once the batch writer flushes, the data is available for read by the
> batch scanner.   On the server side data is read from memory and
> files.
>
> On Wed, Aug 8, 2012 at 1:53 PM,  <[EMAIL PROTECTED]> wrote:
> > I have a set of actors that are doing batchWriters and another set doing
> > batchScanner(s) and I'm getting questions from my users about the time
> > delay between them.  So my question is, what are all the tunable
> > parameters between writes and reads?
> >
> > Do I have the following facts correct?
> >
> > BatchWriters have a time value for flushing to memory cache
> > Minor compaction has its time frequency for flushing to WALogs
> > Major compaction has its time frequency for flushing to disc
> >
> > So where all does a batchScanner get its data from?
> >
> > Any/all help is appreciated.
> >
> >
> > Bob Thorman
> > Engineering Fellow
> > L-3 Communications, ComCept
> > 1700 Science Place
> > Rockwall, TX 75032
> > (972) 772-7501 work
> > [EMAIL PROTECTED]
> > [EMAIL PROTECTED]
> >
> >
>