Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Global Sorting and Multiple Reducers ?

Copy link to this message
Re: Global Sorting and Multiple Reducers ?
At those data sizes, I'd make sure that you have very few maps and reduces.
Look at the input format that blends multiple files together.

If you have multiple reduces, you need the total order partitioner. It
samples the data to see where to cut the key space to make even cuts between
the reducers. You are guaranteed that all keys less than K0 go to reduce 0,
less than K1 go to reduce 1, and so on.

-- Owen