Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: how to execute different tasks on data nodes(simultaneously in hadoop).


Copy link to this message
-
Re: how to execute different tasks on data nodes(simultaneously in hadoop).
[image: Inline image 1]

On Mon, Sep 3, 2012 at 10:01 PM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:

> You can check the value of "map.input.file" in order to apply a different
> logic for each type of files (in the mapper).
> More information about your problem/context would help the readers to
> provide a more extensive reply.
>
> Regards
>
> Bertrand
>
> each data node has to process one equation of above simultaneously.
> On Mon, Sep 3, 2012 at 6:25 PM, Michael Segel <[EMAIL PROTECTED]>wrote:
>
>> Not sure what you are trying to do...
>>
>> You want to pass through the entire data set on all nodes where each node
>> runs a single filter?
>>
>> You're thinking is orthogonal to how Hadoop works.
>>
>> You would be better off letting each node work on a portion of the data
>> which is local to that node running the entire filter set.
>>
>>
>> On Sep 3, 2012, at 11:19 AM, mallik arjun <[EMAIL PROTECTED]> wrote:
>>
>> > genrally in hadoop map function will be exeucted by all the data nodes
>> on the input data set ,against this how can i do the following.
>> > i have some filter programs , and what i want to do is each data
>> node(slave) has to execute one filter alogrithm  simultaneously, diffent
>> from other data nodes executions.
>> >
>> > thanks in advance.
>> >
>> >
>>
>>
>
>
> --
> Bertrand Dechoux
>
+
Bertrand Dechoux 2012-09-03, 17:41
+
Narasingu Ramesh 2012-09-04, 08:03
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB