Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> JobSubmissionFiles: past , present, and future?


+
Jay Vyas 2013-04-12, 11:11
Copy link to this message
-
Re: JobSubmissionFiles: past , present, and future?
To update on this, it was just pointed out to me by matt farrallee
that the auto fix of permissions is for a failsafe
in case of a race condition, and not meant to mend bad permissions in all cases:

https://github.com/apache/hadoop-common/commit/f25dc04795a0e9836e3f237c802bfc1fe8a243ad

Something to keep in mind - if you see the "fixing staging permissions" error message alot
Then there might be a more systemic problem in your fs... At least, that was the case for us.

On Apr 12, 2013, at 6:11 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:

> Hi guys:
>
> I'm curious about the changes and future of the JobSubmissionFiles class.
>
> Grepping around on the web I'm finding some code snippets that suggest that hadoop security is not handled the same way on the staging directory as before:
>
> http://javasourcecode.org/html/open-source/hadoop/hadoop-0.20.203.0/org/apache/hadoop/mapreduce/JobSubmissionFiles.java.html
>
> http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201210.mbox/%3CCAOcnVr0EYLSckXaOCPNm7kBZWpHvCdjBBX5A+[EMAIL PROTECTED]%3E
>
> But I'm having trouble definitively pinning this to versions.
>
> Why the difference in the if/else logic here and what is the future
> Of permissions on .staging?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB