Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> define a specific data node from a reduce task.


Copy link to this message
-
Re: define a specific data node from a reduce task.
Each reduce task take the role of DFS client by using specified output
folder. The placement of reduce
output result judged by NameNode block placement policy. One thing I can
confirm, as the cluster inner
slave, the first block reduce task produced in one DataNode can be placed
into local filesystem.

On Thu, Jun 16, 2011 at 3:21 AM, Pedro Costa <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I'm running an MR application that produces an output that is saved in
> HDFS. My application has 5 slave nodes (so it has also 5 data nodes).
> The hdfs file replication factor is 1. I want from my application, or
> from the Hadoop MR source code tell which data node my result should
> be. For example, if I've datanode1-5, I would like to say to the name
> node to save the reduce output result data in a specific datanode.
>
> Is it possible?
>
>
> Thanks,
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB