Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Re: hadoop.job.ugi backwards compatibility


Copy link to this message
-
Re: hadoop.job.ugi backwards compatibility
On Mon, Sep 13, 2010 at 11:10 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> Yep, but there are plenty of 10 node clusters out there that do important
> work at small startups or single-use-case installations, too. We need to
> provide scalability and security features that work for the 100+ node
> clusters but also not leave the beginners in the dust.

10 node clusters are an important use case, but creating the user
accounts on those clusters is very easy because of the few users.
Futhermore, if the accounts aren't there it just means the users have
no groups. Which for a single use system with security turned off
isn't the end of the world.

> But I think there are plenty of people out there who have built small
> webapps, shell scripts, cron jobs, etc that use hadoop.job.ugi on some
> shared account to impersonate other users.

I'd be surprised. At Yahoo, the primary problem came with people
screen scraping the jobtracker http. With security turned off that
isn't an issue. Again, it isn't hard, just the evolving interface of
UserGroupInformation changed. With security, we tried really hard to
maintain backwards compatibility and succeeded for the vast (99%+)
majority of the users.

> Perhaps I am estimating
> incorrectly - that's why I wanted this discussion on a user-facing list
> rather than a dev-facing list.

Obviously the pointer is there for them to follow into the rabbit hole
of the dev lists. *grin*

> Another example use case that I do a lot on non-secure clusters is: hadoop
> fs -Dhadoop.job.ugi=hadoop,hadoop <something I want to do as a superuser>.
> The permissions model we have in 0.20 obviously isn't secure, but it's nice
> to avoid accidental mistakes, and making it easy to "sudo" like that is
> handy.

It might make sense to add a new switch ( -user ?) to hadoop fs that
does a doAs before doing the
shell command. You could even make it fancy and try to be a proxy user
if security is turned on.

> Regardless of our particular opinions, isn't our policy that we cannot break
> API compatibility between versions without a one-version deprecation period?

There wasn't a way to keep UGI stable. It was a broken design before
the security work. It is marked evolving so we try to minimize
breakage, but it isn't prohibited.

-- Owen