Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack


+
Matt Foley 2012-11-30, 01:51
Copy link to this message
-
Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
Matt,

Let me repost my previous questions and a few more. I'd appreciate your
answers, as it will help me understand the full impact this would have in
Hadoop and related projects.

* Phyton as runtime requirement. Are you planing to migrate all BASH
scripts provided by Hadoop (or dynamically created -ie launcher scripts)
 to Phyton?
* What else in the current build, besides saveVersion.sh, you see as
candidate to be migrated to Phyton?
* How are you planning to define what Phyton modules can be used? Will
developers have to install them manually?
* What kind of tasks you envision Python scripts will enable that are not
possible today?
* Will the requirement of Python be pushed to clients using the hadoop
script? If so, this would affect all downstream projects that use hadoop
script in one why or the other, right?

Is the main motivation of the proposal to make things easier for window, so
there is no need for cygwin? If that is the case, have you considered doing
directly BAT scripts? If you take Tomcat for example, they have BAT scripts
and SH scripts and things work quite nicely.

Personally, I wouldn't be trilled to see the logic in the scripts to get
more complex, but on the opposite direction; IMO, scripts should be trimmed
to set env vars (with no voodoo logic), build the classpath (with no voodoo
logic, just from a set of dirs) and call Java.

Finally, this is code change, so I'm not sure why we are doing a vote.

Thx.

On Thu, Nov 29, 2012 at 3:26 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:

> Matt, thanks for the clarification.
>
> I may have missed the main point of the PROPOSAL thread then. I personally
> want to continue the discussion before voting.
>
> * Phyton as runtime requirement. Are you planing to migrate all BASH
> scripts provided by Hadoop (or dynamically created -ie launcher scripts)
>  to Phyton?
> * What else in the current build, besides saveVersion.sh, you see as
> candidate to be migrated to Phyton?
> * How are you planning to define what Phyton modules can be used? Will
> developers have to install them manually?
>
> Cheers
>
>
> On Thu, Nov 29, 2012 at 2:39 PM, Matt Foley <[EMAIL PROTECTED]>wrote:
>
>> Hi Alejandro,
>> Please see in-line below.
>>
>> On Mon, Nov 26, 2012 at 1:52 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>
>>  wrote:
>>
>> > Matt,
>> >
>> > The scope of this vote seems different from what was discussed in the
>> > PROPOSAL thread.
>> > In the PROPOSAL thread you indicated this was for Hadoop1 because it is
>> ANT
>> > based. And the main reason was to remove saveVersion.sh.
>> > Your #3  was not discussed in the proposal, was it?
>> >
>>
>> The item #3 was in my original statement of the problem, with which I
>> started the proposal thread.  In fact, the thread title was "[PROPOSAL]
>> introduce Python as build-time and run-time dependency for Hadoop and
>> throughout Hadoop stack".  It is true that only one or two people chose to
>> discuss #3 further in that thread.
>>
>> The point is not just to replace a single script, but to provide a means
>> to
>> do cross-platform scripts, which will over time replace many
>> non-platform-specific scripts written in platform-specific languages.
>>
>>
>> >
>> > It seems this vote is dragging much more stuff it was originally
>> discussed.
>> > I think you should suspend the vote, recap the motivation and then
>> restart
>> > the vote.
>> >
>>
>> I respectfully disagree.  I believe a careful reading of the cited
>> discussion thread, plus my own statement of the vote, provides sufficient
>> background for a thoughtful decision on the subject.  Presumably so do the
>> ten other people who had already voted before you made that comment.
>>
>> If several other people want more discussion first, please speak up.
>> Thanks,
>> --Matt
>>
>> As things are laid out at the moment my vote is:
>> >
>> > -1 (It still seems an overkill to introduce a new runtime requirement
>> for
>> > building to replace a script.)
>> > +1 (I think this is the right way to simplify the build)

Alejandro
+
Matt Foley 2012-11-30, 02:26
+
Chuan Liu 2012-11-30, 03:22
+
Bikas Saha 2012-11-30, 04:27
+
Luke Lu 2012-11-30, 11:21
+
Luke Lu 2012-11-30, 12:57
+
Steve Loughran 2012-11-30, 13:29
+
Luke Lu 2012-11-30, 14:02
+
Luke Lu 2012-11-30, 13:49
+
Arun C Murthy 2012-12-02, 18:20
+
Radim Kolar 2012-11-30, 00:29
+
Steve Loughran 2012-11-30, 13:20
+
Radim Kolar 2012-11-30, 13:40
+
Jitendra Pandey 2012-11-30, 22:49
+
Steve Loughran 2012-12-01, 10:48
+
Matt Foley 2012-11-24, 20:13
+
Ivan Mitic 2012-11-29, 23:41
+
Mahadevan Venkatraman 2012-11-30, 02:07
+
Raja Aluri 2012-12-01, 00:57
+
Eli Collins 2012-12-01, 01:08
+
Steve Loughran 2012-12-01, 10:44
+
Doug Cutting 2012-12-01, 18:23
+
Konstantin Boudnik 2012-12-13, 00:53
+
Doug Cutting 2012-11-30, 16:55
+
Joep Rottinghuis 2012-12-01, 20:28
+
Eric Yang 2012-12-02, 06:07
+
Konstantin Boudnik 2012-12-13, 00:55
+
Tom White 2012-12-03, 14:23
+
Chris Nauroth 2012-11-25, 07:18
+
Suresh Srinivas 2012-11-26, 20:41
+
Konstantin Boudnik 2012-11-26, 18:30
+
Radim Kolar 2012-11-26, 17:34
+
Colin McCabe 2012-11-26, 16:53
+
Chris Nauroth 2012-11-26, 17:44
+
Luke Lu 2012-11-26, 17:25
+
Giridharan Kesavan 2012-11-26, 21:16
+
Alejandro Abdelnur 2012-11-26, 21:52
+
Radim Kolar 2012-11-26, 22:17
+
Robert Evans 2012-11-26, 16:16
+
Adam Berry 2012-11-26, 16:45
+
Steve Loughran 2012-11-25, 12:39
+
Doug Cutting 2012-12-03, 18:37
+
Matt Foley 2012-12-03, 19:21
+
Doug Cutting 2012-12-03, 19:37
+
Matt Foley 2012-12-03, 22:08
+
Doug Cutting 2012-12-03, 23:57
+
Matt Foley 2012-12-04, 01:22
+
Doug Cutting 2012-12-04, 04:50
+
Matt Foley 2012-12-04, 17:58
+
Radim Kolar 2012-12-04, 19:41
+
Matt Foley 2012-12-04, 20:28
+
Alejandro Abdelnur 2012-12-04, 21:00
+
Matt Foley 2012-12-04, 22:35
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB