Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> secondary sort - number of reducers


Copy link to this message
-
secondary sort - number of reducers
I have implemented secondary sort in my MR job and for some reason if i
dont specify the number of reducers it uses 1 which doesnt seems right
because im working with 800M+ records and one reducer slows things down
significantly. Is this some kind of limitation with the secondary sort that
it has to use a single reducer .. that kind of would defeat the purpose of
having a scalable solution such as secondary sort. I would appreciate any
help.

Thanks
Adeel