Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Hadoop Multithread MapReduce


Copy link to this message
-
Re: Hadoop Multithread MapReduce
Bertrand Dechoux 2012-07-26, 07:19
Hi,

Summarizing the two previous answers : yes, multithreaded mappers are
supported.
However, the most common use case is to increase the number of map 'slots'.

Have you considered the latter? If so, why would it not help you?
In that case, why do you think multithreading might help you?

In order to understand your need, you need to understand where is your
bottleneck : CPU, local IO, network...

Regards

Bertrand

On Thu, Jul 26, 2012 at 5:13 AM, liu zg <[EMAIL PROTECTED]> wrote:

> hi, ken
> what do you mean multiple threads?
> I think that you can use multiple processes in map task by configuring the
> property of * mapred.tasktracker.map.tasks.maximum* in mapred-site.xml.
>
> Gump
>
>
> On Thu, Jul 26, 2012 at 10:56 AM, kenyh <[EMAIL PROTECTED]> wrote:
>
> >
> > Does anyone know about the feature about using multiple thread in map
> task
> > or
> > reduce task?
> > Is it a good way to use multithread in map task?
> > --
> > View this message in context:
> >
> http://old.nabble.com/Hadoop-Multithread-MapReduce-tp34213534p34213534.html
> > Sent from the Hadoop core-user mailing list archive at Nabble.com.
> >
> >
>

--
Bertrand Dechoux