Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: Namenode memory usage


+
Personal 2013-06-17, 23:20
Copy link to this message
-
RE: Namenode memory usage
Can you take heapdump and check ..? Here you can check which objects are using how much..

Command : jmap -histo:live namenodepid

________________________________
From: Personal [[EMAIL PROTECTED]]
Sent: Tuesday, June 18, 2013 7:20 AM
To: [EMAIL PROTECTED]
Subject: Re: Namenode memory usage
  E Lego

On Mon, Jun 17, 2013 at 1:04 PM, Patai Sangbutsarakum <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

Hi Hadoopers,

My dedicated Namenode box has 48G of memory, 40G is allocated for NN. HEAP

This is from 50070/dfshealth.jsp
28540193 files and directories, 32324098 blocks = 60864291 total. Heap Size is 23.34 GB / 38.54 GB (60%)

The Heap is fluctuating between less than 20G up to almost 100%
However, from top command Residence size is constantly at 39G not matter how low of the memory usage in defshealth.jsp page
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4628 apps      19   0 40.4g  39g  22m S 132.9 83.9  44821:08 /usr/java/jdk/jre/bin/java -Dproc_namenode -Xmx40000m
Is this the time to upgrade the ram to the namenode box?

I remember the easy rule of thumb is 150 bye of every 1M for blocks+file+dir, so 60864291 * 150byte is around 9G. I just don't understand why 40G seems to be used up.?
Please educate..

Hope this make sense
P

+
Patai Sangbutsarakum 2013-06-18, 18:36
+
Chris Nauroth 2013-06-17, 21:56
+
Patai Sangbutsarakum 2013-06-17, 23:12