Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Is perfect control over mapper num AND split distribution possible?


Copy link to this message
-
Re: Is perfect control over mapper num AND split distribution possible?
Seems to work well.  Thank you very much!

On Jan 21, 2014, at 12:42 , Keith Wiley wrote:

> I'll look it up.  Thanks.
>
> On Jan 21, 2014, at 11:43 , java8964 wrote:
>
>> You cannot use hadoop "NLineInputFormat"?
>>
>> If you generate 100 lines of text file, by default, one line will trigger one mapper task.
>>
>> As long as you have 100 task slot available, you will get 100 mapper running concurrently.
>>
>> You want perfect control over mapper num? NLineInputFormat is designed for your purpose.
>>
>> Yong
________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"It's a fine line between meticulous and obsessive-compulsive and a slippery
rope between obsessive-compulsive and debilitatingly slow."
                                           --  Keith Wiley
________________________________________________________________________________