Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> High Ingest on a single server [SEC=UNOFFICIAL]

Copy link to this message
Re: High Ingest on a single server [SEC=UNOFFICIAL]
To verify that this what you're running into, you should see a message
in the master*.debug.log that matches "not balancing because.*"

On 09/01/2013 09:54 PM, John Vines wrote:
> Try restarting the master. A few releases had a big where it would get
> stuck.
> Sent from my phone, please pardon the typos and brevity.
> On Sep 1, 2013 6:12 PM, "Dickson, Matt MR" <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>     __
>     Thanks Eric.
>     The tablet of concern has 2000 tablets while all others have 1000 so
>     it looks like the balancers aren't evening out the tablets per node
>     as expected.
>     Is there a way to force the balancer to run or rectify this
>     situation by moving tablets to alternate nodes?
>     ------------------------------------------------------------------------
>     *From:* Eric Newton [mailto:[EMAIL PROTECTED]
>     <mailto:[EMAIL PROTECTED]>]
>     *Sent:* Thursday, 29 August 2013 23:23
>     *Subject:* Re: High Ingest on a single server [SEC=UNOFFICIAL]
>     The balancers that ship with accumulo attempt to keep an equal
>     number of tablets on each server.  An empty tablet, will be balanced
>     with the same weight as a 50G tablet.
>     You can write a new balancer to take advantage of the properties of
>     the tablets, and any expected hotspots you have.
>     On Thu, Aug 29, 2013 at 1:39 AM, Dickson, Matt MR
>     wrote:
>         __
>         *UNOFFICIAL*
>         We are seeing a single server that has less entries than all the
>         other nodes in the cluster.  Accumulo now appears to be
>         directing higher ingest tablets to this node and its now getting
>         7 times the ingest entries than all other nodes and is slowing
>         or load.  Does Accumulo attempt to balance disk usage across the
>         nodes for a table by moving tablets and that is why we are
>         seeing this node ingesting more?
>         If not, is it possible to make accumulo rebalance the ingest
>         across all servers during a load?
>         Matt