Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - How to control the number of map tasks for each nodes?


Copy link to this message
-
Re: How to control the number of map tasks for each nodes?
edward choi 2010-07-09, 01:42
Mike,

Thanks for the heads up.
Actually, my i7s are 930. Does this make any difference?

Ed

2010/7/9 Michael Segel <[EMAIL PROTECTED]>

>
> Just a nit.
>
> i7s are quad core. While Intel has 2 'threads' per core. (I'm assuming
> you're using the i7's (800 series and below) in a single socket mother
> board.)
> Based on experience with tuning other databases, I am a bit skeptical about
> going beyond the number of cores. Also your memory is probably going to be
> more of a driver on the number of map/reduce tasks you can run
> simultaneously.
>
> JMHO, YMMV.
>
> -Mike
>
>
> > Date: Fri, 9 Jul 2010 10:14:14 +0900
> > Subject: Re: How to control the number of map tasks for each nodes?
> > From: [EMAIL PROTECTED]
> > To: [EMAIL PROTECTED]
> >
> > Thanks you all for the info.
> >
> > I have successfully configured my cluster.
> > Thanks again.
> >
> > 2010/7/8 edward choi <[EMAIL PROTECTED]>
> >
> > > Hi,
> > >
> > > I have a cluster consisting of 11 slaves and a single master.
> > >
> > > The thing is that 3 of my slaves have i7 cpu which means that they can
> have
> > > up to 8 simultaneous processes.
> > > But other slaves only have dual core cpus.
> > >
> > > So I was wondering if I can specify the number of map tasks for each of
> my
> > > slaves.
> > > For example, I want to give 8 map tasks to the slaves that have i7 cpus
> and
> > > only two map tasks to the others.
> > >
> > > Is there a way to do this?
> > >
>
> _________________________________________________________________
> The New Busy is not the old busy. Search, chat and e-mail from your inbox.
>
> http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3