Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Datanodes going down frequently

Copy link to this message
Re: Datanodes going down frequently
You have only 2 GB of ram? Have you cheked if you are swapping?


Sent from my iPad

On Sep 15, 2011, at 4:27 PM, Aaron Baff <[EMAIL PROTECTED]> wrote:

> Do they eventually recover and get added back into the cluster? Do you have a ton of blocks? I've noticed that sometimes the block checker will take so long, and will tie up so much CPU (and memory, which hits a GC cycle) that it stops reporting to the NN for a while, but when it finishes its check, it resumes talking to the NN and the NN adds it back in.
> --Aaron
> -----Original Message-----
> From: john smith [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, September 15, 2011 3:07 PM
> Subject: Datanodes going down frequently
> Hi all,
> I am running a 10 node cluster (1NN + 9DN, ubuntu server 10.04, 2GB RAM
> each). I am facing a strange problem. My datanodes go down randomly and
> nothing showup in the logs. They lose their network connectivity suddenly
> and NN declares them as dead. Any one faced this problem? Is it because of
> hadoop or is it some problem with my infrastructure?
> The worst part of the problem is, I need to manually go to the remote
> machine and restart networking. Can someone help me with this? Did any one
> face a similar kind of a problem
> Btw: my had version : 0.20.2
> Thanks,
> jS