Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Is perfect control over mapper num AND split distribution possible?


Copy link to this message
-
Re: Is perfect control over mapper num AND split distribution possible?
Keith Wiley 2014-01-21, 20:42
I'll look it up.  Thanks.

On Jan 21, 2014, at 11:43 , java8964 wrote:

> You cannot use hadoop "NLineInputFormat"?
>
> If you generate 100 lines of text file, by default, one line will trigger one mapper task.
>
> As long as you have 100 task slot available, you will get 100 mapper running concurrently.
>
> You want perfect control over mapper num? NLineInputFormat is designed for your purpose.
>
> Yong

________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"What I primarily learned in grad school is how much I *don't* know.
Consequently, I left grad school with a higher ignorance to knowledge ratio than
when I entered."
                                           --  Keith Wiley
________________________________________________________________________________