Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> the same key in different reducers


Copy link to this message
-
the same key in different reducers
Hi ,
My hadoop job writes results of map/reduce to HBase.
I have 3 reducers.

Here is a sequence of input and output parameters for Mapper , Combiner and
Reducer
    *input: InputFormat<K1,V1>
    mapper: Mapper<K1,V1,K2,V2>
    combiner: Reducer<K2,V2,K2,V2>
    reducer: Reducer<K2,V2,K3,V3>
    output: RecordWriter<K3,V3>

*My question:
  Is it possible that more than one reducer has the same output key K3.
Meaning in case I have 3 reducers is it possible that
reducer1    K3 -* 1* , V3 [1,2,3]
reducer2    K3 - 2 , V3 [5,6,9]
reducer3    K3 - *1* , V3 [10,15,22]

As you can see reducer1 has K3 - 1 and reducer3 has K3 - 1.
So is that case possible or every and every reducer has unique output key?

Thanks in advance
Oleg.
+
Ted Yu 2010-06-09, 21:06
+
James Seigel 2010-06-09, 21:40
+
Ted Yu 2010-06-09, 21:43
+
Alex Kozlov 2010-06-09, 22:15
+
Owen OMalley 2010-06-10, 02:30
+
Oleg Ruchovets 2010-06-10, 16:27
+
Owen OMalley 2010-06-09, 21:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB