Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack


Copy link to this message
-
Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
Steve Loughran 2012-11-25, 12:39
On 24 November 2012 20:13, Matt Foley <[EMAIL PROTECTED]> wrote:

> For discussion, please see previous thread "[PROPOSAL] introduce Python as
> build-time and run-time dependency for Hadoop and throughout Hadoop stack".
>
> This vote consists of three separate items:
>
> 1. Contributors shall be allowed to use Python as a platform-independent
> scripting language for build-time tasks, and add Python as a build-time
> dependency.
> Please vote +1, 0, -1.
>
>
+1

> 2. Contributors shall be encouraged to use Maven tasks in combination with
> either plug-ins or Groovy scripts to do cross-platform build-time tasks,
> even under ant in Hadoop-1.
> Please vote +1, 0, -1.
>
>
+1

My feelings on Maven are well known, but Groovy can mitigate things. And
I'm not going to advocate post-M2 build tools such as Gradle.

It's ironic that Maven's utter inflexibility forces people to use scripting
languages to get their work done, but Groovy is fairly nimble here -and
easy to learn for any Java programmer. "Groovy in Action" is the book to
own.

> 3. Contributors shall be allowed to use Python as a platform-independent
> scripting language for run-time tasks, and add Python as a run-time
> dependency.
> Please vote +1, 0, -1.
>
+1. I look forward to never having to debug shell script env variable
inheritance ever again.

This does not mean that I advocate writing big bits of the system in .py;
as someone who is debugging OpenStack request throttling this weekend, I
know that Python is not "the solution" to problems. For Hadoop it has a
role, but the role should be ('better than bash') and ('streaming
integration').
> Note that voting -1 on #1 and +1 on #2 essentially REQUIRES contributors to
> use Maven plug-ins or Groovy as the only means of cross-platform build-time
> tasks, or to simply continue using platform-dependent scripts as is being
> done today.
>
> Vote closes at 12:30pm PST on Saturday 1 December.
> ---------
> Personally, my vote is +1, +1, +1.
> I think #2 is preferable to #1, but still has many unknowns in it, and
> until those are worked out I don't want to delay moving to cross-platform
> scripts for build-time tasks.
>
> Best regards,
> --Matt
>