Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Namenode memory usage


Copy link to this message
-
Namenode memory usage
Patai Sangbutsarakum 2013-06-17, 20:03
Hi Hadoopers,

My dedicated Namenode box has 48G of memory, 40G is allocated for NN. HEAP

This is from 50070/dfshealth.jsp
*28540193 files and directories, 32324098 blocks = 60864291 total. Heap
Size is 23.34 GB / 38.54 GB (60%) *

*The Heap is fluctuating between less than 20G up to almost 100%*
*
*
*
*
However, from top command Residence size is constantly at 39G not matter
how low of the memory usage in defshealth.jsp page
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

 4628 apps      19   0 40.4g  39g  22m S 132.9 83.9  44821:08
/usr/java/jdk/jre/bin/java -Dproc_namenode -Xmx40000m
Is this the time to upgrade the ram to the namenode box?

I remember the easy rule of thumb is 150 bye of every 1M for
blocks+file+dir, so *60864291 * 150byte is around 9G. I just don't
understand why 40G seems to be used up.?*
*Please educate..*
*
*
*Hope this make sense*
*P*