Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Namenode memory usage


Copy link to this message
-
Namenode memory usage
Hi Hadoopers,

My dedicated Namenode box has 48G of memory, 40G is allocated for NN. HEAP

This is from 50070/dfshealth.jsp
*28540193 files and directories, 32324098 blocks = 60864291 total. Heap
Size is 23.34 GB / 38.54 GB (60%) *

*The Heap is fluctuating between less than 20G up to almost 100%*
*
*
*
*
However, from top command Residence size is constantly at 39G not matter
how low of the memory usage in defshealth.jsp page
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

 4628 apps      19   0 40.4g  39g  22m S 132.9 83.9  44821:08
/usr/java/jdk/jre/bin/java -Dproc_namenode -Xmx40000m
Is this the time to upgrade the ram to the namenode box?

I remember the easy rule of thumb is 150 bye of every 1M for
blocks+file+dir, so *60864291 * 150byte is around 9G. I just don't
understand why 40G seems to be used up.?*
*Please educate..*
*
*
*Hope this make sense*
*P*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB