Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> question for understanding partitioning

Copy link to this message
Re: question for understanding partitioning
On 01/18/2011 03:09 PM, Mapred Learn wrote:
> hi,
> I have a basic question. How does partitioning work ?
> Following is a scenario I created to put up my question.
> i) A parttition function is defined as partitioning map-output based on
> aphabetical sorting of the key i.e. a partition for keys starting with 'a',
> partition for keys starting with 'b'... partition for keys starting with
> 'z'. So, it means each map may have atmost 26 partitions ?
> ii) What input will Reducer get ? Reducer will get first partition
> (partition starting with 'a') of all the maps as it's input ? Does it mean
> we will need 26 reduce tasks ?
> Any inputs/documents/examples on this are appreciated. I am bit confused by
> this.
> Thanks in advance

You should probably read the Yahoo tutorial to brush up on the topic
before asking on the list.


If you still don't understand after that, and you post a specific
question (i.e., not "how does partitioning work") I'm sure someone will
be able to answer.