Marc Sturlese 2013-08-22, 07:41
-Re: rack awarness unexpected behaviour
Nicolas Liochon 2013-08-22, 08:21
Do the jobs run on the whole cluster or a single rack?
If you write from a single rack, you will get something similar to what you
described, because the default policy is to put one block locally and 2
blocks on the same remote rack. It does check that there is enough place
available, but does not try to balance.
On Thu, Aug 22, 2013 at 9:41 AM, Marc Sturlese <[EMAIL PROTECTED]>wrote:
> Hey there,
> I've set up rack awareness on my hadoop cluster with replication 3. I have
> racks and each contains 50% of the nodes.
> I can see that the blocks are spread on the 2 racks, the problem is that
> nodes from a rack are storing 2 replicas and the nodes of the other rack
> just one. If I launch the hadoop balancer script, it will properly spread
> the replicas across the 2 racks, leaving all nodes with exactly the same
> available disk space but, after jobs are running for hours, the data will
> unbalanced again (rack1 having all nodes with less empty disk space than
> nodes from rack2)
> Any clue whats going on?
> Thanks in advance
> View this message in context:
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
Marc Sturlese 2013-08-22, 08:29
Nicolas Liochon 2013-08-22, 08:57
Marc Sturlese 2013-08-22, 09:20
Harsh J 2013-08-22, 10:14
Michel Segel 2013-08-22, 10:57
Jun Ping Du 2013-08-23, 04:29
Marc Sturlese 2013-10-03, 07:52
Michael Segel 2013-10-03, 12:23
Jun Ping Du 2013-10-04, 01:11
Marc Sturlese 2013-10-03, 13:58
Michel Segel 2013-10-03, 15:53