Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Is there a way to limit # of hadoop tasks per user at runtime?


Copy link to this message
-
Is there a way to limit # of hadoop tasks per user at runtime?
Hi Everyone,

I was wondering if there is a way for limiting the number of tasks (map+reduce) *per user* at runtime? Using an environment variable perhaps? I am asking this from a resource provisioning perspective. I am trying to come up with a N-token licensing system for multiple users to use our limited hadoop resources simultaneously. That is, when user A checks out 6 tokens,  he/she can only run 6 hadoop tasks.

If there is no such thing in hadoop, has anyone tried to integrate hadoop with torque/moab (or any other RM or scheduler)? Any advice in that direction will be appreciated :)

Thanks in advance,
-Mehmet