Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Any mechanism in Hadoop to run in background


Copy link to this message
-
Re: Any mechanism in Hadoop to run in background
Yes, you can change your task tracker startup script to use nice and ionice
and restart the task tracker process. The mappers and reducers spun off
this task tracker will inherit the niceness.

See the first comment in
http://blog.cloudera.com/blog/2011/04/hbase-dos-and-donts/
Quoting:
change the hadoop-0.20-tasktracker so the process is started like this:

daemon *nice -n 19 ionice -c2 -n7*/usr/lib/hadoop-0.20/bin/hadoop-daemon.sh –
config “/etc/hadoop-0.20/conf” start tasktracker $DAEMON_FLAGS

--S

On Sat, Jun 22, 2013 at 7:55 AM, yun peng <[EMAIL PROTECTED]> wrote:

> Hi, All...
> We have a user case intended to run Mapreduce in background, while the
> server serves online operations. The MapReduce job may have lower priority
> comparing to the online jobs..
>
> I know this is a different use case of Mapreduce comparing to its
> originally targeted scenario (where Mapreduce largely own resource
> exclusively)... But I want to know if there is any tuning knobs that allow
> Mapreduce to run in low priority/with limited resource.
>
> Thanks,
> Yun
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB