Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Is the sort(in sort and shuffle) always required


Copy link to this message
-
Is the sort(in sort and shuffle) always required
Hello,
My question: is the sort (in the sort and shuffle) absolutely required?
If I wanted mapreduce to partition (using the map) and then aggregate(using
reduce) without a need for the keys to be sorted
is it possible to turn of the sorting? Or is the fact that keys come to the
reducer in sorted order just a side effect of sorting and that
the sorting is vital for the efficient operation of MapReduce?
Thanks
Saptarshi
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB