Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - -m Number of Mappers


Copy link to this message
-
Re: -m Number of Mappers
Abraham Elmahrek 2013-12-13, 22:53
Nick,

Could you provide your sqoop command?

Also, checkout the Sqoop Cookbook on a really good description on how
--num-mappers option works:
http://books.google.com/books?id=3qKAW063BhoC&pg=PA19&lpg=PA19&dq=controlling+parallelism+sqoop&source=bl&ots=HWYeBk36l5&sig=Cy7-AOHQO3ON7R0Kv4Pobdj-FNg&hl=en&sa=X&ei=oo-rUuGXNYrmoAS5sIK4BQ&ved=0CDIQ6AEwAQ#v=onepage&q=controlling%20parallelism%20sqoop&f=false
.
On Tue, Dec 10, 2013 at 6:36 PM, Martin, Nick <[EMAIL PROTECTED]> wrote:

>  Kind of a newbie question but one that’s been stumping me…
>
>
>
> I’m noticing when I set the number of mappers to use (i.e. –m in CLI Sqoop
> command) that isn’t the number of mappers my cluster is using to execute
> the Sqoop job.
>
>
>
> For instance, I just ran an import with –m 8 specified and the number of
> mappers used is 4 until the import hits 50% and then drops to 3 for the
> rest of the import process (my cluster has 16 available map slots; no other
> concurrent jobs processing).
>
>
>
> I read through the “Controlling Parallelism” piece of the user guide but
> didn’t seem to find my answer.
>
>
>
> Thanks in advanced for the help,
>
> Nick
>
>
>
>
>