-Re: Global Sorting and Multiple Reducers ?
Owen O'Malley 2010-11-12, 18:15
At those data sizes, I'd make sure that you have very few maps and reduces.
Look at the input format that blends multiple files together.
If you have multiple reduces, you need the total order partitioner. It
samples the data to see where to cut the key space to make even cuts between
the reducers. You are guaranteed that all keys less than K0 go to reduce 0,
less than K1 go to reduce 1, and so on.