Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

Matt Foley 2012-11-21, 19:15
Radim Kolar 2012-11-23, 23:40
Matt Foley 2012-11-24, 20:13
Radim Kolar 2012-11-24, 21:26
Konstantin Boudnik 2012-11-24, 22:03
Alejandro Abdelnur 2012-11-21, 19:25
Radim Kolar 2012-11-21, 20:46
Konstantin Boudnik 2012-11-21, 21:33
Konstantin Boudnik 2012-11-21, 20:00
Matt Foley 2012-11-21, 21:14
Konstantin Boudnik 2012-11-21, 21:50
Andy Isaacson 2012-11-21, 23:00
Radim Kolar 2012-11-21, 23:58
Steve Loughran 2012-11-22, 09:21
Konstantin Boudnik 2012-11-22, 01:46
Radim Kolar 2012-11-22, 01:57
Chris Nauroth 2012-11-21, 21:03
Radim Kolar 2012-11-21, 21:30
Chris Nauroth 2012-11-21, 21:44
Radim Kolar 2012-11-21, 23:15
Chris Nauroth 2012-11-22, 00:14
Radim Kolar 2012-11-22, 01:55
Copy link to this message
Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
Chris Nauroth 2012-11-22, 02:40
This predates me, so I don't know the rationale for repackaging Tomcat
inside HTTPFS.  I suspect that there was a desire to create a fully
stand-alone distribution package, including a full web server.  The Maven
Jetty plugin isn't directly applicable to this use case.  I don't know why
it was decided to use Tomcat instead of Jetty.  (If anyone else out there
has the background, please respond.)  Regardless, if the desire is to
package a full web server instead of just the war, then switching to Jetty
would not change the challenges of the build process.  We'd still need to
preserve whatever permissions are present in the Jetty distribution.

In general, when I was working on this, I did not question whether the
current packaging was "correct".  I assumed that whatever changes I made
for Windows compatibility must yield the exact same distribution without
changes on currently supported platforms like Linux.  If there are
questions around actually changing the output of the build process, then
that will steer the conversation in another direction and increase the
scope of this effort.

It seems like the trickiest issue is preservation of permissions and
symlinks in tar files.  I suspect that any JVM-based solution like custom
Maven plugins, Groovy, or jtar would be limited in this respect.  According
to Ant documentation, it's a JDK limitation, so I suspect all of these
would have the same problem.  I haven't tried any of them though.  (If
there was a feasible solution, then Ant likely would have incorporated it
long ago.)  If anyone wants to try though, we might learn something from

Thank you,
On Wed, Nov 21, 2012 at 5:55 PM, Radim Kolar <[EMAIL PROTECTED]> wrote:

> Dne 22.11.2012 1:14, Chris Nauroth napsal(a):
>  The trickiest maintenance issue is hadoop-hdfs-httpfs, where we unpack
>> and repack a Tomcat.
> why its not possible to just ship WAR file? Its seems to be special
> purpose app and they needs hand security setup anyway and intergration with
> existing firewall/web infrastructure.
> did you considered to use Jetty? it has really good maven support:
> http://wiki.eclipse.org/Jetty/**Feature/Jetty_Maven_Plugin<http://wiki.eclipse.org/Jetty/Feature/Jetty_Maven_Plugin>
> I am using jetty 8 instead of tomcat and run it with java -jar start.jar
> no extra file permissions like x bit are needed.
> If you really need to create tar by hand, there is java library for doing
> it - http://code.google.com/p/jtar/ and it can be used from any JVM based
> script language, you have plenty of choices.
Radim Kolar 2012-11-22, 14:54
Steve Loughran 2012-11-22, 09:02
Matt Foley 2012-11-21, 19:44
Alejandro Abdelnur 2012-11-21, 19:58
Steve Loughran 2012-11-22, 09:14