-
Re: how to execute different tasks on data nodes(simultaneously in hadoop).Michael Segel 2012-09-03, 16:25
Not sure what you are trying to do...
You want to pass through the entire data set on all nodes where each node runs a single filter?
You're thinking is orthogonal to how Hadoop works.
You would be better off letting each node work on a portion of the data which is local to that node running the entire filter set.
On Sep 3, 2012, at 11:19 AM, mallik arjun <[EMAIL PROTECTED]> wrote:
> genrally in hadoop map function will be exeucted by all the data nodes on the input data set ,against this how can i do the following.
> i have some filter programs , and what i want to do is each data node(slave) has to execute one filter alogrithm simultaneously, diffent from other data nodes executions.
>
> thanks in advance.
>
>
You want to pass through the entire data set on all nodes where each node runs a single filter?
You're thinking is orthogonal to how Hadoop works.
You would be better off letting each node work on a portion of the data which is local to that node running the entire filter set.
On Sep 3, 2012, at 11:19 AM, mallik arjun <[EMAIL PROTECTED]> wrote:
> genrally in hadoop map function will be exeucted by all the data nodes on the input data set ,against this how can i do the following.
> i have some filter programs , and what i want to do is each data node(slave) has to execute one filter alogrithm simultaneously, diffent from other data nodes executions.
>
> thanks in advance.
>
>
