Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Is there a way to limit # of hadoop tasks per user at runtime?


Copy link to this message
-
Is there a way to limit # of hadoop tasks per user at runtime?
Hi Everyone,

I was wondering if there is a way for limiting the number of tasks (map+reduce) *per user* at runtime? Using an environment variable perhaps? I am asking this from a resource provisioning perspective. I am trying to come up with a N-token licensing system for multiple users to use our limited hadoop resources simultaneously. That is, when user A checks out 6 tokens,  he/she can only run 6 hadoop tasks.

If there is no such thing in hadoop, has anyone tried to integrate hadoop with torque/moab (or any other RM or scheduler)? Any advice in that direction will be appreciated :)

Thanks in advance,
-Mehmet
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB