-Re: Increase number of reducers for bulk data load to empty HBase table
Karthik Ranganathan 2011-10-19, 15:41
The only way to increase the number of reducers is to have more regions -
each reducer produces an output per region, so the number of reducers =number of regions.
On 10/18/11 2:00 AM, "Matthew Tovbin" <[EMAIL PROTECTED]> wrote:
>I'm willing to bulk load data from hdfs folders into HBase, for this
>I used configureIncrementalLoad method from HFileOutputFormat that
>configures the job, as follows:
>The problem is that destination table in HBase is empty, meaning it's only
>hosted by one region server, so the resulted number of reducers is 1,
>makes the job to run almost forever.
>How can I increase the number of reducers? Can the number of reducers be
>to more than a number of region servers?
>Thanks in advance,
> Matthew Tovbin.