Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Hadoop tool-kit for monitoring


Copy link to this message
-
Re: Hadoop tool-kit for monitoring
Konstantin Boudnik 2011-05-17, 22:16
Also, it seems like Ganglia would be very well complemented by Nagios
to allow you to monitor an overall health of your cluster.
--
  Take care,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622

Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.

On Tue, May 17, 2011 at 15:15, Allen Wittenauer <[EMAIL PROTECTED]> wrote:
>
> On May 17, 2011, at 3:11 PM, Mark question wrote:
>
>> So what other memory consumption tools do you suggest? I don't want to do it
>> manually and dump statistics into file because IO will affect performance
>> too.
>
>        We watch memory with Ganglia.  We also tune our systems such that a task will only take X amount.  In other words, given an 8gb RAM:
>
>        1gb for the OS
>        1gb for the TT and DN
>        6gb for all tasks
>
>        if we assume each task will take max 1gb, then we end up with 3 maps and 3 reducers.
>
>        Keep in mind that the mem consumed is more than just JVM heap size.