Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> How to assign a mapreduce process to a specific pool


Copy link to this message
-
Re: How to assign a mapreduce process to a specific pool
> I'm using `hadoop jar` command
not sure it does mean you know how to specify the pool through hadoop
jar command already, just in case.

./bin/hadoop jar hadoop-examples-0.20.2-cdh3u0.jar wordcount
-Dmapred.fairscheduler.pool=log2hive input output

hth
p

On Tue, Oct 25, 2011 at 8:57 AM, Raimon Bosch <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I'm trying to launch a process to map our log data into hive tables and I
> was trying to assign this job to a specific pool name. This is our
> configuration file:
>
> --------- allocations.xml --------
> <?xml version="1.0"?>
> <allocations>
>
>  <pool name="log2hive">
>    <minMaps>14</minMaps>
>    <minReduces>6</minReduces>
>    <maxRunningJobs>1</maxRunningJobs>
>    <minSharePreemptionTimeout>300</minSharePreemptionTimeout>
>    <weight>1.0</weight>
>  </pool>
>
>  <defaultMinSharePreemptionTimeout>600</defaultMinSharePreemptionTimeout>
>  <fairSharePreemptionTimeout>600</fairSharePreemptionTimeout>
>
> </allocations>
> --------- mapred-site.xml --------------
>  <property>
>    <name>mapred.jobtracker.taskScheduler</name>
>    <value>org.apache.hadoop.mapred.FairScheduler</value>
>  </property>
>
>  <property>
>    <name>mapred.fairscheduler.allocation.file</name>
>    <value>/hadoop/conf/allocations.xml</value>
>  </property>
>
> My question is how can I use this new pool to launch jobs? I'm using `hadoop
> jar` command but I would be also interested in assign pools from my code.
>
>
> Thanks in advance,
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB