Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - the same key in different reducers


Copy link to this message
-
the same key in different reducers
Oleg Ruchovets 2010-06-09, 08:17
Hi ,
My hadoop job writes results of map/reduce to HBase.
I have 3 reducers.

Here is a sequence of input and output parameters for Mapper , Combiner and
Reducer
    *input: InputFormat<K1,V1>
    mapper: Mapper<K1,V1,K2,V2>
    combiner: Reducer<K2,V2,K2,V2>
    reducer: Reducer<K2,V2,K3,V3>
    output: RecordWriter<K3,V3>

*My question:
  Is it possible that more than one reducer has the same output key K3.
Meaning in case I have 3 reducers is it possible that
reducer1    K3 -* 1* , V3 [1,2,3]
reducer2    K3 - 2 , V3 [5,6,9]
reducer3    K3 - *1* , V3 [10,15,22]

As you can see reducer1 has K3 - 1 and reducer3 has K3 - 1.
So is that case possible or every and every reducer has unique output key?

Thanks in advance
Oleg.