Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - NLineInputFormat and very high number of maptasks


Copy link to this message
-
Re: NLineInputFormat and very high number of maptasks
Saptarshi Guha 2009-01-20, 16:42
Sorry, i see - every line is now a maptask - one split,one task.(in  
this case N=1 line per split)
Is that correct?
Saptarshi

On Jan 20, 2009, at 11:39 AM, Saptarshi Guha wrote:

> Hello,
> When I use NLIneInputFormat, when I output:
>
> System
> .out.println("mapred.map.tasks:"+jobConf.get("mapred.map.tasks"));
> I see 51, but on the jobtracker site, the number is 18114. Yet with
> TextInputFormat it shows 51.
> I'm using Hadoop - 0.19
>
> Any ideas why?
> Regards
> Saptarshi
>
> --
> Saptarshi Guha - [EMAIL PROTECTED]

Saptarshi Guha | [EMAIL PROTECTED] | http://www.stat.purdue.edu/~sguha
If the church put in half the time on covetousness that it does on lust,
this would be a better world.
-- Garrison Keillor, "Lake Wobegon Days"