Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> question for understanding partitioning


Copy link to this message
-
Re: question for understanding partitioning
On 01/18/2011 03:09 PM, Mapred Learn wrote:
> hi,
> I have a basic question. How does partitioning work ?
>
> Following is a scenario I created to put up my question.
>
> i) A parttition function is defined as partitioning map-output based on
> aphabetical sorting of the key i.e. a partition for keys starting with 'a',
> partition for keys starting with 'b'... partition for keys starting with
> 'z'. So, it means each map may have atmost 26 partitions ?
>
> ii) What input will Reducer get ? Reducer will get first partition
> (partition starting with 'a') of all the maps as it's input ? Does it mean
> we will need 26 reduce tasks ?
>
> Any inputs/documents/examples on this are appreciated. I am bit confused by
> this.
>
> Thanks in advance
>

You should probably read the Yahoo tutorial to brush up on the topic
before asking on the list.

http://developer.yahoo.com/hadoop/tutorial/module5.html#partitioning

If you still don't understand after that, and you post a specific
question (i.e., not "how does partitioning work") I'm sure someone will
be able to answer.

DR
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB