-Re: Sizing walog area
Terry P. 2013-10-24, 16:58
Just to ensure I'm looking at the correct property, by "in-memory map size
per tserver" are you referring to the property *tserver.memory.maps.max* in
*accumulo-site.xml*? If that's the case, I'm using 1GB for that property.
I am running loggers on each tserver. Is the default that Accumulo writes
to at least two loggers? I see config item *
logger.recovery.file.replication* is set to 2 (by default); is that what
Digging around, I also see that logger.archive.replication is set to 2, and
logger.archive is false. What does logger.archive and
logger.archive.replication do? I find no mention of "archive" in the User
Many thanks Christopher, your help is always appreciated.
On Wed, Oct 23, 2013 at 2:10 PM, Christopher <[EMAIL PROTECTED]> wrote:
> The data in the write-ahead logs is needed until the tserver flushes
> the in memory maps to disk. Assuming you have a logger running on
> every tserver, and tservers write to at least two loggers, you should
> ensure that the size of the disk area is *at least* two times as big
> as your in-memory map size per tserver. I'd say 5x-10x the in-memory
> map size is probably safe. So, if your tservers are running with 2GB
> of memory, then a 10-20GB area is probably more than sufficient.
> Christopher L Tubbs II
> On Wed, Oct 23, 2013 at 1:02 PM, Terry P. <[EMAIL PROTECTED]> wrote:
> > Greetings all,
> > For Accumulo 1.4 where write ahead logs are not yet stored in HDFS, does
> > anyone have guidancewith respect to sizing the walog area? What exactly
> > triggers when write ahead logs get removed? What might cause them to
> > around for an extended period of time (as in under abnormal
> > The system this applies to will see an ingest rate of approximately 2000
> > docs per second averaging 1-2K each (broken out into 12 columns each, so
> > 24,000 entries per second) across 6 tabletserver nodes.
> > Thanks in advance,
> > Terry