Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Is perfect control over mapper num AND split distribution possible?


Copy link to this message
-
Re: Is perfect control over mapper num AND split distribution possible?
Seems to work well.  Thank you very much!

On Jan 21, 2014, at 12:42 , Keith Wiley wrote:

> I'll look it up.  Thanks.
>
> On Jan 21, 2014, at 11:43 , java8964 wrote:
>
>> You cannot use hadoop "NLineInputFormat"?
>>
>> If you generate 100 lines of text file, by default, one line will trigger one mapper task.
>>
>> As long as you have 100 task slot available, you will get 100 mapper running concurrently.
>>
>> You want perfect control over mapper num? NLineInputFormat is designed for your purpose.
>>
>> Yong
________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"It's a fine line between meticulous and obsessive-compulsive and a slippery
rope between obsessive-compulsive and debilitatingly slow."
                                           --  Keith Wiley
________________________________________________________________________________
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB