Hello,
When I use NLIneInputFormat, when I output:
System.out.println("mapred.map.tasks:"+jobConf.get("mapred.map.tasks"));
I see 51, but on the jobtracker site, the number is 18114. Yet with
TextInputFormat it shows 51.
I'm using Hadoop - 0.19

Any ideas why?
Regards
Saptarshi

--
Saptarshi Guha - [EMAIL PROTECTED]

Sorry, i see - every line is now a maptask - one split,one task.(in  
this case N=1 line per split)
Is that correct?
Saptarshi

On Jan 20, 2009, at 11:39 AM, Saptarshi Guha wrote:

> Hello,
> When I use NLIneInputFormat, when I output:
>
> System
> .out.println("mapred.map.tasks:"+jobConf.get("mapred.map.tasks"));
> I see 51, but on the jobtracker site, the number is 18114. Yet with
> TextInputFormat it shows 51.
> I'm using Hadoop - 0.19
>
> Any ideas why?
> Regards
> Saptarshi
>
> --
> Saptarshi Guha - [EMAIL PROTECTED]

Saptarshi Guha | [EMAIL PROTECTED] | http://www.stat.purdue.edu/~sguha
If the church put in half the time on covetousness that it does on lust,
this would be a better world.
-- Garrison Keillor, "Lake Wobegon Days"

Saptarshi Guha wrote:
> Sorry, i see - every line is now a maptask - one split,one task.(in
> this case N=1 line per split)
> Is that correct?
> Saptarshi
>
You are right. NLineInputFormat splits N lines of input as one split and
each split is given to a map task.
By default, N is 1. N can configured through the property
"mapred.line.input.format.linespermap".
> On Jan 20, 2009, at 11:39 AM, Saptarshi Guha wrote:
>
>> Hello,
>> When I use NLIneInputFormat, when I output:
>>     System.out.println("mapred.map.tasks:"+jobConf.get("mapred.map.tasks"));
>>
Where are you printing this statement? Looks like the JobConf, that you
are looking at, is not set with the correct value of number of map tasks
yet.
>> I see 51, but on the jobtracker site, the number is 18114. Yet with
>> TextInputFormat it shows 51.
>> I'm using Hadoop - 0.19
>>
>> Any ideas why?
>> Regards
>> Saptarshi
>>
>> --Saptarshi Guha - [EMAIL PROTECTED]
>
> Saptarshi Guha | [EMAIL PROTECTED] |
> http://www.stat.purdue.edu/~sguha
> If the church put in half the time on covetousness that it does on lust,
> this would be a better world.
>         -- Garrison Keillor, "Lake Wobegon Days"
>

-Amareshwari