Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> -m Number of Mappers


Copy link to this message
-
Re: -m Number of Mappers
Nick,

Could you provide your sqoop command?

Also, checkout the Sqoop Cookbook on a really good description on how
--num-mappers option works:
http://books.google.com/books?id=3qKAW063BhoC&pg=PA19&lpg=PA19&dq=controlling+parallelism+sqoop&source=bl&ots=HWYeBk36l5&sig=Cy7-AOHQO3ON7R0Kv4Pobdj-FNg&hl=en&sa=X&ei=oo-rUuGXNYrmoAS5sIK4BQ&ved=0CDIQ6AEwAQ#v=onepage&q=controlling%20parallelism%20sqoop&f=false
.
On Tue, Dec 10, 2013 at 6:36 PM, Martin, Nick <[EMAIL PROTECTED]> wrote:

>  Kind of a newbie question but one that’s been stumping me…
>
>
>
> I’m noticing when I set the number of mappers to use (i.e. –m in CLI Sqoop
> command) that isn’t the number of mappers my cluster is using to execute
> the Sqoop job.
>
>
>
> For instance, I just ran an import with –m 8 specified and the number of
> mappers used is 4 until the import hits 50% and then drops to 3 for the
> rest of the import process (my cluster has 16 available map slots; no other
> concurrent jobs processing).
>
>
>
> I read through the “Controlling Parallelism” piece of the user guide but
> didn’t seem to find my answer.
>
>
>
> Thanks in advanced for the help,
>
> Nick
>
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB