Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - MapReduce: Reducers partitions.


Copy link to this message
-
Re: MapReduce: Reducers partitions.
Stack 2013-04-11, 00:33
On Wed, Apr 10, 2013 at 12:01 PM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi Greame,
>
> No. The reducer will simply write on the table the same way you are doing a
> regular Put. If a split is required because of the size, then the region
> will be split, but at the end, there will not necessary be any region
> split.
>
> In the usecase described below, all the 600 lines will "simply" go into the
> only region in the table and no split will occur.
>
> The goal is to partition the data for the reducer only. Not in the table.
>
Then just use the default partitioner?

The suggestion that you use HTablePartitioner seems inappropriate to your
task.  See the sink doc here:
http://hadoop.apache.org/docs/r2.0.3-alpha/api/org/apache/hadoop/mapreduce/lib/partition/HashPartitioner.html

St.Ack