-Re: slow transfer rates w/ HDFS balancer
Allen Wittenauer 2009-10-08, 19:06
On 10/8/09 12:03 PM, "David Howell" <[EMAIL PROTECTED]> wrote:
> I just added a couple of new node to my cluster today, and ran the
> balancer to get some data onto them. With scp, the transfer rate
> between these machines is in the neighborhood of 50 MB/s... but the
> transfers during the balancing cap out at 1 MB/s.
> Is this to be expected, and if not, does anyone have any ideas of
> where to start looking for a problem?
Balancer was intended to run as a background job so that the grid isn't
limping while the balance is going on. But this is a tunable: change the