Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Sorting/Partitioning of Pig output


Copy link to this message
-
Sorting/Partitioning of Pig output
I understand in the traditional map/reduce paradigm that each key will get sent to the same reducer sorted but in pig there is no such thing as a "key".  I'm curious to know how pig knows to which reducer to send its output to?

So when creating a custom StoreFunc is there any guarentee on the ordering of Tuples that come into putNext?

And another even more basic question. Do StoreFuncs operate at the Map phase or Reduce phase?

Thanks
+
Jonathan Coveney 2013-03-27, 20:41
+
Yen SYU 2013-03-28, 16:23
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB