Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> NLineInputFormat and very high number of maptasks


Copy link to this message
-
Re: NLineInputFormat and very high number of maptasks
Sorry, i see - every line is now a maptask - one split,one task.(in  
this case N=1 line per split)
Is that correct?
Saptarshi

On Jan 20, 2009, at 11:39 AM, Saptarshi Guha wrote:

> Hello,
> When I use NLIneInputFormat, when I output:
>
> System
> .out.println("mapred.map.tasks:"+jobConf.get("mapred.map.tasks"));
> I see 51, but on the jobtracker site, the number is 18114. Yet with
> TextInputFormat it shows 51.
> I'm using Hadoop - 0.19
>
> Any ideas why?
> Regards
> Saptarshi
>
> --
> Saptarshi Guha - [EMAIL PROTECTED]

Saptarshi Guha | [EMAIL PROTECTED] | http://www.stat.purdue.edu/~sguha
If the church put in half the time on covetousness that it does on lust,
this would be a better world.
-- Garrison Keillor, "Lake Wobegon Days"
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB