Does anyone know about the feature about using multiple thread in map task or
reduce task?
Is it a good way to use multithread in map task?
--
View this message in context: http://old.nabble.com/Hadoop-Multithread-MapReduce-tp34213534p34213534.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Hi,

We do have a Multithreaded Mapper implementation available for use.
Check out: http://hadoop.apache.org/common/docs/stable/api/org/apache/hadoop/mapreduce/lib/map/MultithreadedMapper.html

On Thu, Jul 26, 2012 at 8:26 AM, kenyh <[EMAIL PROTECTED]> wrote:
>
> Does anyone know about the feature about using multiple thread in map task or
> reduce task?
> Is it a good way to use multithread in map task?
> --
> View this message in context: http://old.nabble.com/Hadoop-Multithread-MapReduce-tp34213534p34213534.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>

--
Harsh J

hi, ken
what do you mean multiple threads?
I think that you can use multiple processes in map task by configuring the
property of * mapred.tasktracker.map.tasks.maximum* in mapred-site.xml.

Gump
On Thu, Jul 26, 2012 at 10:56 AM, kenyh <[EMAIL PROTECTED]> wrote:

>
> Does anyone know about the feature about using multiple thread in map task
> or
> reduce task?
> Is it a good way to use multithread in map task?
> --
> View this message in context:
> http://old.nabble.com/Hadoop-Multithread-MapReduce-tp34213534p34213534.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>

Hi,

Summarizing the two previous answers : yes, multithreaded mappers are
supported.
However, the most common use case is to increase the number of map 'slots'.

Have you considered the latter? If so, why would it not help you?
In that case, why do you think multithreading might help you?

In order to understand your need, you need to understand where is your
bottleneck : CPU, local IO, network...

Regards

Bertrand

On Thu, Jul 26, 2012 at 5:13 AM, liu zg <[EMAIL PROTECTED]> wrote:

> hi, ken
> what do you mean multiple threads?
> I think that you can use multiple processes in map task by configuring the
> property of * mapred.tasktracker.map.tasks.maximum* in mapred-site.xml.
>
> Gump
>
>
> On Thu, Jul 26, 2012 at 10:56 AM, kenyh <[EMAIL PROTECTED]> wrote:
>
> >
> > Does anyone know about the feature about using multiple thread in map
> task
> > or
> > reduce task?
> > Is it a good way to use multithread in map task?
> > --
> > View this message in context:
> >
> http://old.nabble.com/Hadoop-Multithread-MapReduce-tp34213534p34213534.html
> > Sent from the Hadoop core-user mailing list archive at Nabble.com.
> >
> >
>

--
Bertrand Dechoux