Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Mapper and Reducer takes longer than usual for a HBase table aggregation task


Copy link to this message
-
Re: Mapper and Reducer takes longer than usual for a HBase table aggregation task
Pavan:
Did you use TableInputFormat or its variant ?
If so, take a look at TableSplit and how it is used in
TableInputFormatBase#getSplits().

Cheers
On Sun, Aug 25, 2013 at 2:36 PM, Jens Scheidtmann <
[EMAIL PROTECTED]> wrote:

> Hi Pavan,
>
>
>> 2. ) If my table is in the order of millions, the number of mappers is
>> increased to 5.. How does Hadoop know how many mappers to run for a
>> specific job?
>>
>> The number of input splits determines the number of mappers. Usually (in
> the default case) your source is split into hdfs blocks (usually 64 MB) and
> for each block, there will be a mapper.
>
> Best regards,
>
> Jens
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB