Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Re: Hadoop Installation Mappers setting


Copy link to this message
-
Re: Hadoop Installation Mappers setting
Jitendra Yadav 2013-05-23, 18:56
Hi,

Thanks for your clarification.

I have one more question.

How does cores factor influence slots calculation?

Thanks~

On 5/23/13, Amal G Jose <[EMAIL PROTECTED]> wrote:
> I am explaining it more.
> If your machine have 8 GB of memory.
> After reserving to Operating system and all other processes except
> tasktracker, you have 4 GB remaining(assume).
> The remaining process running is tasktracker.
> If the child jvm size is 200 MB,
> Then you can define a maximum slots of 4*1024 MB/ 200 MB
> Which is approximately 20.
> You can divide the slots into mapper and reducer slots as per your
> requirement.
> This is just an example that I explained based on my knowledge.
>
>
>
> On Thu, May 23, 2013 at 7:48 PM, <[EMAIL PROTECTED]> wrote:
>
>> **
>> Hi
>>
>> I assume the question is on how many slots.
>>
>> It dependents on
>> - the child/task jvm size and the available memory.
>> - available number of cores
>>
>>
>>
>> Your available memory for tasks is total memory - memory used for OS and
>> other services running on your box.
>>
>> Other services include non hadoop services as well as hadoop daemons.
>>
>> Divide the available memory with child jvm size and that would get the
>> max
>> num of slots.
>>
>> Also check whether sufficient number of cores are available as well.
>>
>>
>> Regards
>> Bejoy KS
>>
>> Sent from remote device, Please excuse typos
>> ------------------------------
>> *From: * Jitendra Yadav <[EMAIL PROTECTED]>
>> *Date: *Thu, 23 May 2013 18:10:38 +0530
>> *To: *<[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Hadoop Installation Mappers setting
>>
>> Hi,
>>
>> While installing hadoop cluster, how we can calculate the exact number of
>> mappers value.
>>
>>
>> Thanks~
>>
>
+
bejoy.hadoop@... 2013-05-23, 19:15