Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Hadoop Installation Mappers setting


Copy link to this message
-
Re: Hadoop Installation Mappers setting
Hi,

Thanks for your clarification.

I have one more question.

How does cores factor influence slots calculation?

Thanks~

On 5/23/13, Amal G Jose <[EMAIL PROTECTED]> wrote:
> I am explaining it more.
> If your machine have 8 GB of memory.
> After reserving to Operating system and all other processes except
> tasktracker, you have 4 GB remaining(assume).
> The remaining process running is tasktracker.
> If the child jvm size is 200 MB,
> Then you can define a maximum slots of 4*1024 MB/ 200 MB
> Which is approximately 20.
> You can divide the slots into mapper and reducer slots as per your
> requirement.
> This is just an example that I explained based on my knowledge.
>
>
>
> On Thu, May 23, 2013 at 7:48 PM, <[EMAIL PROTECTED]> wrote:
>
>> **
>> Hi
>>
>> I assume the question is on how many slots.
>>
>> It dependents on
>> - the child/task jvm size and the available memory.
>> - available number of cores
>>
>>
>>
>> Your available memory for tasks is total memory - memory used for OS and
>> other services running on your box.
>>
>> Other services include non hadoop services as well as hadoop daemons.
>>
>> Divide the available memory with child jvm size and that would get the
>> max
>> num of slots.
>>
>> Also check whether sufficient number of cores are available as well.
>>
>>
>> Regards
>> Bejoy KS
>>
>> Sent from remote device, Please excuse typos
>> ------------------------------
>> *From: * Jitendra Yadav <[EMAIL PROTECTED]>
>> *Date: *Thu, 23 May 2013 18:10:38 +0530
>> *To: *<[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Hadoop Installation Mappers setting
>>
>> Hi,
>>
>> While installing hadoop cluster, how we can calculate the exact number of
>> mappers value.
>>
>>
>> Thanks~
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB