-Re: Multi-threaded map task
Mark Olimpiati 2013-01-14, 07:22
Thanks for the reply Nitin, but I don't see what's the bottleneck of having
it distributed with multi-threaded maps ?
I see your point in that each map is processing different splits, but my
question is if each map task had 2 threads multiplexing or running in
parallel if there is enough cores to process the same split, wouldn't that
be faster with enough cores?
On Sun, Jan 13, 2013 at 10:34 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
> Thats because its distributed processing framework over network
> On Jan 14, 2013 11:27 AM, "Mark Olimpiati" <[EMAIL PROTECTED]> wrote:
> > Hi, this is a simple question, but why wasn't map or reduce tasks
> > programmed to be multi-threaded ? ie. instead of spawning 6 map tasks
> for 6
> > cores, run one map task with 6 parallel threads.
> > In fact I tried this myself, but turns that threading is not helping as
> > would be in regular java programs for some reason .. any feedback on this
> > topic?
> > Thanks,
> > Mark