Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Mapper and Reducer takes longer than usual for a HBase table aggregation task


Copy link to this message
-
Re: Mapper and Reducer takes longer than usual for a HBase table aggregation task
Pavan:
Did you use TableInputFormat or its variant ?
If so, take a look at TableSplit and how it is used in
TableInputFormatBase#getSplits().

Cheers
On Sun, Aug 25, 2013 at 2:36 PM, Jens Scheidtmann <
[EMAIL PROTECTED]> wrote:

> Hi Pavan,
>
>
>> 2. ) If my table is in the order of millions, the number of mappers is
>> increased to 5.. How does Hadoop know how many mappers to run for a
>> specific job?
>>
>> The number of input splits determines the number of mappers. Usually (in
> the default case) your source is split into hdfs blocks (usually 64 MB) and
> for each block, there will be a mapper.
>
> Best regards,
>
> Jens
>
>