-Is there a way to limit # of hadoop tasks per user at runtime?
Mehmet Belgin 2013-05-21, 21:43
I was wondering if there is a way for limiting the number of tasks (map+reduce) *per user* at runtime? Using an environment variable perhaps? I am asking this from a resource provisioning perspective. I am trying to come up with a N-token licensing system for multiple users to use our limited hadoop resources simultaneously. That is, when user A checks out 6 tokens, he/she can only run 6 hadoop tasks.
If there is no such thing in hadoop, has anyone tried to integrate hadoop with torque/moab (or any other RM or scheduler)? Any advice in that direction will be appreciated :)
Thanks in advance,