Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Sending data to all reducers


Copy link to this message
-
Re: Sending data to all reducers
So you are trying to run a single reducer on each machine, and all input
data regardless of its location gets streamed to each reducer?

On Thu, Aug 23, 2012 at 10:41 AM, Hamid Oliaei <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I want to broadcast some data to all nodes under Hadoop 0.20.2. I tested
> DistributedCache module. Unfortunately, it was time-consuming
> and runtime is important for my work.
> I want to write a MR job so that a copy of input data are generated in
> output of all reducers.
> Is that possible? How?
> I mean I want to have copies of some data to the number of reducers.
>
> Thanks,
>
> Hamid Oliaei
>
> [EMAIL PROTECTED]
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB