Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack


+
Matt Foley 2012-11-21, 19:15
+
Radim Kolar 2012-11-23, 23:40
+
Matt Foley 2012-11-24, 20:13
+
Radim Kolar 2012-11-24, 21:26
+
Konstantin Boudnik 2012-11-24, 22:03
+
Alejandro Abdelnur 2012-11-21, 19:25
+
Radim Kolar 2012-11-21, 20:46
+
Konstantin Boudnik 2012-11-21, 21:33
+
Konstantin Boudnik 2012-11-21, 20:00
+
Matt Foley 2012-11-21, 21:14
+
Konstantin Boudnik 2012-11-21, 21:50
+
Andy Isaacson 2012-11-21, 23:00
+
Radim Kolar 2012-11-21, 23:58
+
Steve Loughran 2012-11-22, 09:21
+
Konstantin Boudnik 2012-11-22, 01:46
+
Radim Kolar 2012-11-22, 01:57
+
Chris Nauroth 2012-11-21, 21:03
+
Radim Kolar 2012-11-21, 21:30
+
Chris Nauroth 2012-11-21, 21:44
+
Radim Kolar 2012-11-21, 23:15
+
Chris Nauroth 2012-11-22, 00:14
+
Radim Kolar 2012-11-22, 01:55
+
Chris Nauroth 2012-11-22, 02:40
+
Radim Kolar 2012-11-22, 14:54
Copy link to this message
-
Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
Steve Loughran 2012-11-22, 09:02
On 22 November 2012 02:40, Chris Nauroth <[EMAIL PROTECTED]> wrote:

>
> It seems like the trickiest issue is preservation of permissions and
> symlinks in tar files.  I suspect that any JVM-based solution like custom
> Maven plugins, Groovy, or jtar would be limited in this respect.  According
> to Ant documentation, it's a JDK limitation, so I suspect all of these
> would have the same problem.  I haven't tried any of them though.  (If
> there was a feasible solution, then Ant likely would have incorporated it
> long ago.)  If anyone wants to try though, we might learn something from
> that.
>
> Thank you,
> --Chris
>
>
You are limited by what File.canRead(), canWrite() and canExecute) tell you.

The absence of a way to detect file permissions in Java -is because of the
lowest-common-denominator approach of the JavaFS APIs, supporting FAT32
(odd case logic, no perms or symlinks), NTFS (odd case logic, ACLs over
perms, symlinks historically very hard to create), HFS+ (case insensitive
unix fs!) as well as classic unixy filesystems.

Ant <tarfileset> filesets in <tar> let you spec permissions on filesets you
pull into the tar; they are generated x-platform, which the other reason
why you declare them in <tar> -you have the right to generate proper tar
files even if you use a Windows box.

symlinks are problematic -even detecting them cross platform is pretty
unreliable. To really do them you'd need to add a new <symlinkfileset>
entity for <tar>, that would take the link declaration. I could imagine how
to do that -and if stuck into the hadoop tools JAR, wouldn't even depend on
a new version of Ant.

Maven just adds extra layers in the way.

-Steve
+
Matt Foley 2012-11-21, 19:44
+
Alejandro Abdelnur 2012-11-21, 19:58
+
Steve Loughran 2012-11-22, 09:14