Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> WritableComparable

Copy link to this message
Hi Folks,

After much poking around I am still unable to determine why I am seeing
'reduce' being called twice with the "same" key.
Recall from my previous email that "sameness" is determined by 'compareTo'
of my custom key type.

AFAIK, the default WritableComparator invokes 'compareTo' for any two keys
which are being ordered during sorting and merging.
Is it somehow possible that a bitwise comparator is used for the spilled map
output rather than the default WritableComparator?

I am out of clues, short of studying the "shuffling" code.  If anyone can
suggest some further debugging steps, don't be shy. :)