Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> WritableComparable


Copy link to this message
-
WritableComparable
Hi Folks,

After much poking around I am still unable to determine why I am seeing
'reduce' being called twice with the "same" key.
Recall from my previous email that "sameness" is determined by 'compareTo'
of my custom key type.

AFAIK, the default WritableComparator invokes 'compareTo' for any two keys
which are being ordered during sorting and merging.
Is it somehow possible that a bitwise comparator is used for the spilled map
output rather than the default WritableComparator?

I am out of clues, short of studying the "shuffling" code.  If anyone can
suggest some further debugging steps, don't be shy. :)

Thanks!!!

stan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB