Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Mapper basic question


Copy link to this message
-
Re: Mapper basic question
Hi  Tariq \Arun,

The no of blocks(splits) = *total no of file size/hdfs block size *
replicate value*
The no of splits is again nothing but the blocks here.

Other than increasing the block size(input splits) is it possible to limit
that no of mappers?
Cheers!
Manoj.

On Wed, Jul 11, 2012 at 6:06 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> Take a look at CombineFileInputFormat - this will create 'meta splits'
> which include multiple small spilts, thus reducing #maps which are run.
>
> Arun
>
> On Jul 11, 2012, at 5:29 AM, Manoj Babu wrote:
>
> Hi,
>
> The no of mappers is depends on the no of blocks. Is it possible to limit
> the no of mappers size without increasing the HDFS block size?
>
> Thanks in advance.
>
> Cheers!
> Manoj.
>
>
>  --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB