-RE: Namenode memory usage
Brahma Reddy Battula 2013-06-18, 01:45
Can you take heapdump and check ..? Here you can check which objects are using how much..
Command : jmap -histo:live namenodepid
From: Personal [[EMAIL PROTECTED]]
Sent: Tuesday, June 18, 2013 7:20 AM
To: [EMAIL PROTECTED]
Subject: Re: Namenode memory usage
On Mon, Jun 17, 2013 at 1:04 PM, Patai Sangbutsarakum <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
My dedicated Namenode box has 48G of memory, 40G is allocated for NN. HEAP
This is from 50070/dfshealth.jsp
28540193 files and directories, 32324098 blocks = 60864291 total. Heap Size is 23.34 GB / 38.54 GB (60%)
The Heap is fluctuating between less than 20G up to almost 100%
However, from top command Residence size is constantly at 39G not matter how low of the memory usage in defshealth.jsp page
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4628 apps 19 0 40.4g 39g 22m S 132.9 83.9 44821:08 /usr/java/jdk/jre/bin/java -Dproc_namenode -Xmx40000m
Is this the time to upgrade the ram to the namenode box?
I remember the easy rule of thumb is 150 bye of every 1M for blocks+file+dir, so 60864291 * 150byte is around 9G. I just don't understand why 40G seems to be used up.?
Hope this make sense