Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - Minor compaction occurring often with fairly long delays during ingest.


Copy link to this message
-
Re: Minor compaction occurring often with fairly long delays during ingest.
Keith Turner 2013-06-26, 12:09
Its possible that you could be running into ACCUMULO-1062 [1], although I
am thinking your large values may be at issue.  Erics suggestion of having
more tablets can help work around this issue, in addition to allowing more
minc parallelism.

[1] : https://issues.apache.org/jira/browse/ACCUMULO-1062
On Tue, Jun 25, 2013 at 7:06 PM, Hider, Sandy <[EMAIL PROTECTED]>wrote:

> I recently setup Accumulo 1.4.2 on a rack of boxes that each has 24
> processors and 43 GB of RAM.  I set them up using the 3GB example templates
> but then increased the max size of the Tserver and a few other components
> to 5GB.
>
> Doing some initial tests importing roughly 7000 records, each record has
> approximately 7 small fields and 1 large field holding data between 200Kb
> to 1Mb in size.  While ingesting I am seeing the server hold and start
> minor compactions which seem to take quite a while after 2000-3000 records,
> and then occurring again fairly frequently
>
> I am wondering what options I have to try and minimize the frequency of
> minor compactions during ingest.    What components memory sizes and config
> properties would help me avoid this problem?  If anyone has other ideas for
> me to try and fix this please let me know.
>
> Thanks in advance,
>
> Sandy
>
>
>
>