Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Is perfect control over mapper num AND split distribution possible?


+
Keith Wiley 2014-01-21, 19:28
+
java8964 2014-01-21, 19:43
Copy link to this message
-
Re: Is perfect control over mapper num AND split distribution possible?
I'll look it up.  Thanks.

On Jan 21, 2014, at 11:43 , java8964 wrote:

> You cannot use hadoop "NLineInputFormat"?
>
> If you generate 100 lines of text file, by default, one line will trigger one mapper task.
>
> As long as you have 100 task slot available, you will get 100 mapper running concurrently.
>
> You want perfect control over mapper num? NLineInputFormat is designed for your purpose.
>
> Yong

________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"What I primarily learned in grad school is how much I *don't* know.
Consequently, I left grad school with a higher ignorance to knowledge ratio than
when I entered."
                                           --  Keith Wiley
________________________________________________________________________________
+
Keith Wiley 2014-01-21, 21:42
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB