Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack


Copy link to this message
-
Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
Steve Loughran 2012-11-22, 09:02
On 22 November 2012 02:40, Chris Nauroth <[EMAIL PROTECTED]> wrote:

>
> It seems like the trickiest issue is preservation of permissions and
> symlinks in tar files.  I suspect that any JVM-based solution like custom
> Maven plugins, Groovy, or jtar would be limited in this respect.  According
> to Ant documentation, it's a JDK limitation, so I suspect all of these
> would have the same problem.  I haven't tried any of them though.  (If
> there was a feasible solution, then Ant likely would have incorporated it
> long ago.)  If anyone wants to try though, we might learn something from
> that.
>
> Thank you,
> --Chris
>
>
You are limited by what File.canRead(), canWrite() and canExecute) tell you.

The absence of a way to detect file permissions in Java -is because of the
lowest-common-denominator approach of the JavaFS APIs, supporting FAT32
(odd case logic, no perms or symlinks), NTFS (odd case logic, ACLs over
perms, symlinks historically very hard to create), HFS+ (case insensitive
unix fs!) as well as classic unixy filesystems.

Ant <tarfileset> filesets in <tar> let you spec permissions on filesets you
pull into the tar; they are generated x-platform, which the other reason
why you declare them in <tar> -you have the right to generate proper tar
files even if you use a Windows box.

symlinks are problematic -even detecting them cross platform is pretty
unreliable. To really do them you'd need to add a new <symlinkfileset>
entity for <tar>, that would take the link declaration. I could imagine how
to do that -and if stuck into the hadoop tools JAR, wouldn't even depend on
a new version of Ant.

Maven just adds extra layers in the way.

-Steve