Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> MapReduce: Reducers partitions.


+
Jean-Marc Spaggiari 2013-04-10, 12:49
+
Nitin Pawar 2013-04-10, 12:55
+
Jean-Marc Spaggiari 2013-04-10, 13:54
+
Ted Yu 2013-04-10, 13:58
+
Nitin Pawar 2013-04-10, 14:01
+
Jean-Marc Spaggiari 2013-04-10, 14:03
+
Jean-Marc Spaggiari 2013-04-10, 18:26
+
Graeme Wallace 2013-04-10, 18:52
+
Jean-Marc Spaggiari 2013-04-10, 19:01
Copy link to this message
-
Re: MapReduce: Reducers partitions.
On Wed, Apr 10, 2013 at 12:01 PM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi Greame,
>
> No. The reducer will simply write on the table the same way you are doing a
> regular Put. If a split is required because of the size, then the region
> will be split, but at the end, there will not necessary be any region
> split.
>
> In the usecase described below, all the 600 lines will "simply" go into the
> only region in the table and no split will occur.
>
> The goal is to partition the data for the reducer only. Not in the table.
>
Then just use the default partitioner?

The suggestion that you use HTablePartitioner seems inappropriate to your
task.  See the sink doc here:
http://hadoop.apache.org/docs/r2.0.3-alpha/api/org/apache/hadoop/mapreduce/lib/partition/HashPartitioner.html

St.Ack
+
Jean-Marc Spaggiari 2013-04-11, 11:52
+
Ted Yu 2013-04-11, 00:03
+
Graeme Wallace 2013-04-10, 19:06
+
Stack 2013-04-11, 00:26