Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - How to assign a mapreduce process to a specific pool


Copy link to this message
-
Re: How to assign a mapreduce process to a specific pool
Patai Sangbutsarakum 2011-10-25, 17:13
> I'm using `hadoop jar` command
not sure it does mean you know how to specify the pool through hadoop
jar command already, just in case.

./bin/hadoop jar hadoop-examples-0.20.2-cdh3u0.jar wordcount
-Dmapred.fairscheduler.pool=log2hive input output

hth
p

On Tue, Oct 25, 2011 at 8:57 AM, Raimon Bosch <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I'm trying to launch a process to map our log data into hive tables and I
> was trying to assign this job to a specific pool name. This is our
> configuration file:
>
> --------- allocations.xml --------
> <?xml version="1.0"?>
> <allocations>
>
>  <pool name="log2hive">
>    <minMaps>14</minMaps>
>    <minReduces>6</minReduces>
>    <maxRunningJobs>1</maxRunningJobs>
>    <minSharePreemptionTimeout>300</minSharePreemptionTimeout>
>    <weight>1.0</weight>
>  </pool>
>
>  <defaultMinSharePreemptionTimeout>600</defaultMinSharePreemptionTimeout>
>  <fairSharePreemptionTimeout>600</fairSharePreemptionTimeout>
>
> </allocations>
> --------- mapred-site.xml --------------
>  <property>
>    <name>mapred.jobtracker.taskScheduler</name>
>    <value>org.apache.hadoop.mapred.FairScheduler</value>
>  </property>
>
>  <property>
>    <name>mapred.fairscheduler.allocation.file</name>
>    <value>/hadoop/conf/allocations.xml</value>
>  </property>
>
> My question is how can I use this new pool to launch jobs? I'm using `hadoop
> jar` command but I would be also interested in assign pools from my code.
>
>
> Thanks in advance,
>