Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - Does anyone know how to get a sqoop job into a specific fair scheduler pool?


Copy link to this message
-
Re: Does anyone know how to get a sqoop job into a specific fair scheduler pool?
David Medinets 2013-05-29, 14:59
Add the following to mapred-site.xml. Note that you can change the set of
queue names.

<property>
  <name>mapred.fairscheduler.poolnameproperty</name>
  <value>mapred.job.queue.name</value>
</property>
<property>
  <name>mapred.queue.names</name>
  <value>default,sqoop</value>
</property>

Define queues in the Fair Scheduler configuration file. They are
defined the same as
pools ... just use <queue> instead of <pool>.

Lastly, and this is the tricky part, use -Dmapred.job.queue.name *after* sqoop
import but *before* any other options.

sqoop import -Dmapred.job.queue.name=sqoop --user=ME --password=etc...

Unless I've forgotten something, that's all you need. It might be
important to know
that pools and queues seem to be separate. If you use the above
configuration, you
probably should always specify the job's queue name.

p.s. - is anyone else annoyed by the new gmail email editor?

On Wed, May 29, 2013 at 8:55 AM, Billie Rinaldi <[EMAIL PROTECTED]>wrote:

> Nope.  The fair scheduler docs talk about how to configure which property
> determines the pool.  Have you looked at that?
> http://hadoop.apache.org/docs/stable/fair_scheduler.html
>
>
> On Wed, May 29, 2013 at 5:48 AM, David Medinets <[EMAIL PROTECTED]>wrote:
>
>> This is accumulo-related because the data is destined for Accumulo. :)
>>
>
>