Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> How are 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' calculated?


+
hadoop qi 2013-07-11, 16:47
+
Shahab Yunus 2013-07-12, 13:47
+
Vinod Kumar Vavilapalli 2013-07-12, 17:24
+
hadoop qi 2013-07-12, 20:47
Copy link to this message
-
Re: How are 'PHYSICAL_MEMORY_BYTES' and 'VIRTUAL_MEMORY_BYTES' calculated?
As Vinod Kumar Vavilapalli they are indeed snapshots in point and time. So
they are neither the peak usage from the whole duration of the job, nor
cumulative aggregate that increases over time.

Regards,
Shahab
On Fri, Jul 12, 2013 at 4:47 PM, hadoop qi <[EMAIL PROTECTED]> wrote:

> Thanks for the response. So they represent the total physical memory
> (virtual memory) has been allocated to the job (e.g., from heap and stack)
> during its entire life time? I am still confused how to get the cumulative
> number from /proc/meminfo. I think from /proc/meminfo we can only get the
> memory usage of a  process in a particular time point (looked like a
> snapshot of the status of the process). If these numbers are added, the sum
> would be much more than memory allocated to the program.
>
>
> On Fri, Jul 12, 2013 at 6:47 AM, Shahab Yunus <[EMAIL PROTECTED]>wrote:
>
>> I think they are cumulative but per task.
>>
>> Physical memory bytes
>> (PHYSICAL_MEMORY_BYTES)
>> The physical memory being used by a task in bytes, as reported by
>> /proc/meminfo.
>> Virtual memory bytes
>> (VIRTUAL_MEMORY_BYTES)
>> The virtual memory being used by a task in bytes, as reported by
>> /proc/meminfo.
>>
>> This is from the Definitive Guide book. Page 260.
>>
>> Regards,
>> Shhab
>>
>>
>> On Thu, Jul 11, 2013 at 12:47 PM, hadoop qi <[EMAIL PROTECTED]> wrote:
>>
>>> Hello,
>>>
>>> I am wondering how memory counters  'PHYSICAL_MEMORY_BYTES'  and
>>> 'VIRTUAL_MEMORY_BYTES'  are calculated? They are peaks of memory usage or
>>> cumulative usage?
>>>
>>> Thanks for help,
>>>
>>
>>
>
+
Vinod Kumar Vavilapalli 2013-07-13, 00:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB