Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> -m Number of Mappers


Copy link to this message
-
-m Number of Mappers
Kind of a newbie question but one that's been stumping me...

I'm noticing when I set the number of mappers to use (i.e. -m in CLI Sqoop command) that isn't the number of mappers my cluster is using to execute the Sqoop job.

For instance, I just ran an import with -m 8 specified and the number of mappers used is 4 until the import hits 50% and then drops to 3 for the rest of the import process (my cluster has 16 available map slots; no other concurrent jobs processing).

I read through the "Controlling Parallelism" piece of the user guide but didn't seem to find my answer.

Thanks in advanced for the help,
Nick
+
Abraham Elmahrek 2013-12-13, 22:53
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB