Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Any mechanism in Hadoop to run in background


+
yun peng 2013-06-22, 14:55
Copy link to this message
-
Re: Any mechanism in Hadoop to run in background
Yes, you can change your task tracker startup script to use nice and ionice
and restart the task tracker process. The mappers and reducers spun off
this task tracker will inherit the niceness.

See the first comment in
http://blog.cloudera.com/blog/2011/04/hbase-dos-and-donts/
Quoting:
change the hadoop-0.20-tasktracker so the process is started like this:

daemon *nice -n 19 ionice -c2 -n7*/usr/lib/hadoop-0.20/bin/hadoop-daemon.sh –
config “/etc/hadoop-0.20/conf” start tasktracker $DAEMON_FLAGS

--S

On Sat, Jun 22, 2013 at 7:55 AM, yun peng <[EMAIL PROTECTED]> wrote:

> Hi, All...
> We have a user case intended to run Mapreduce in background, while the
> server serves online operations. The MapReduce job may have lower priority
> comparing to the online jobs..
>
> I know this is a different use case of Mapreduce comparing to its
> originally targeted scenario (where Mapreduce largely own resource
> exclusively)... But I want to know if there is any tuning knobs that allow
> Mapreduce to run in low priority/with limited resource.
>
> Thanks,
> Yun
>