Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Set num of mappers in a TableMapReduceUtil.initTableMapperJob


Copy link to this message
-
Re: Set num of mappers in a TableMapReduceUtil.initTableMapperJob
The short answer is that it picks the mappers count is the locked to the
number of regions for the table you're working against using that
inputformat since this is most efficient.  You would have to make your own
inputformat if you want to split on something other than regions.

On Wed, Feb 29, 2012 at 5:52 PM, Doug Meil <[EMAIL PROTECTED]>wrote:

>
> You probably want to see this...
>
> http://hbase.apache.org/book.html#splitter
>
>
>
>
>
>
> On 2/29/12 7:48 PM, "Vrushali C" <[EMAIL PROTECTED]> wrote:
>
> >
> >
> >I am using TableMapReduceUtil.initTableMapperJob to initiate a map reduce
> >job that scans the entire table and processes records in it.
> >
> >I wanted to know if I can control/hint to the framework for setting the
> >number of mappers.
> >
> >I was searching online and I see that JobConf's conf.setNumMapTasks(int
> >num) is deprecated.  I tried setting the number of tasks in the code
> >c.set("mapred.map.tasks", "5"); or
> >
> >    c.setInt("mapred.map.tasks", 5);
> >
> >but that doesn't work.
> >
> >
> >thanks
> >Vrushali
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB