-Re: Is perfect control over mapper num AND split distribution possible?
Keith Wiley 2014-01-21, 21:42
Seems to work well. Thank you very much!
On Jan 21, 2014, at 12:42 , Keith Wiley wrote:
> I'll look it up. Thanks.
> On Jan 21, 2014, at 11:43 , java8964 wrote:
>> You cannot use hadoop "NLineInputFormat"?
>> If you generate 100 lines of text file, by default, one line will trigger one mapper task.
>> As long as you have 100 task slot available, you will get 100 mapper running concurrently.
>> You want perfect control over mapper num? NLineInputFormat is designed for your purpose.
Keith Wiley [EMAIL PROTECTED] keithwiley.com music.keithwiley.com
"It's a fine line between meticulous and obsessive-compulsive and a slippery
rope between obsessive-compulsive and debilitatingly slow."
-- Keith Wiley