Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: how to execute different tasks on data nodes(simultaneously in hadoop).


Copy link to this message
-
Re: how to execute different tasks on data nodes(simultaneously in hadoop).
[image: Inline image 1]

On Mon, Sep 3, 2012 at 10:01 PM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:

> You can check the value of "map.input.file" in order to apply a different
> logic for each type of files (in the mapper).
> More information about your problem/context would help the readers to
> provide a more extensive reply.
>
> Regards
>
> Bertrand
>
> each data node has to process one equation of above simultaneously.
> On Mon, Sep 3, 2012 at 6:25 PM, Michael Segel <[EMAIL PROTECTED]>wrote:
>
>> Not sure what you are trying to do...
>>
>> You want to pass through the entire data set on all nodes where each node
>> runs a single filter?
>>
>> You're thinking is orthogonal to how Hadoop works.
>>
>> You would be better off letting each node work on a portion of the data
>> which is local to that node running the entire filter set.
>>
>>
>> On Sep 3, 2012, at 11:19 AM, mallik arjun <[EMAIL PROTECTED]> wrote:
>>
>> > genrally in hadoop map function will be exeucted by all the data nodes
>> on the input data set ,against this how can i do the following.
>> > i have some filter programs , and what i want to do is each data
>> node(slave) has to execute one filter alogrithm  simultaneously, diffent
>> from other data nodes executions.
>> >
>> > thanks in advance.
>> >
>> >
>>
>>
>
>
> --
> Bertrand Dechoux
>