Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Bigtop, mail # dev - [DISCUSS] BOM for release 0.7.0 of Bigtop


+
Konstantin Boudnik 2013-07-09, 03:50
+
Andrew Purtell 2013-07-09, 15:59
+
Sean Mackrory 2013-07-09, 16:47
+
Konstantin Boudnik 2013-07-09, 17:11
Copy link to this message
-
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Bruno Mahé 2013-07-09, 17:10
On 07/09/2013 09:47 AM, Sean Mackrory wrote:
> Without wanting to detract from the spirit of focussing on system
> stability, I'd like to suggest a few changes I think it's time we at least
> discuss seriously:
>
> JDKs: I've seen a lot of people ask about JDK 7. Perhaps time to add
> support for Oracle JDK 7? It's working pretty well in my experience, and
> although it's less tested upstream, the only JDK we officially support is
> officially EOL, so we're not exactly in a good position now IMO.
>
> Debian 7 has also been out for a while, and I think we should do at least
> one release on it. It's likely very little work but I think there's value
> in certifying the stack will work well there. (On the topic of OS's - are
> we specifically talking SP3 of SLES 11?). I don't feel strongly on this,
> but I'm just curious if there's a reason you're suggesting staying with
> 12.10 and not 13.04 - other than wanting less change in this release?
> Again, I hardly have an opinion on that one.
>
> Other components that have recently had releases that I don't consider to
> impact the
> - Hue 2.4.0
> - Whirr 0.8.2
> - Flume 1.4.0
>
> There's also been a ticket to package Avro for a long time and I'd like to
> get to that soon. Perhaps Parquet as well? Although like Phoenix and
> DataFu, I would suggest doing just the libraries for now, not all the CLI
> tools.
>
> Again - I don't mean to take away from the focus on stability, but I also
> don't think we shouldn't stretch to stay up to date either.
>
> +1 to everything else as suggested, however.
>
>
> On Tue, Jul 9, 2013 at 8:59 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:
>> Would you be willing to consider Phoenix, only BIGTOP-993? Installing the
>> package produced by 993 only drops a library for HBase into
>> /usr/lib/phoenix, essentially the same relationship between the DataFu
>> package and Pig. There is follow up work that is more ambitious, for
>> example BIGTOP-1007, but that is not required by any means and could come
>> in if/whenever you are comfortable with it.
>>
>>
>> On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
>>
>>> Guys,
>>>
>>> I wanna kick-off the discussion on the content of 0.7.0 BOM
>>> Release 0.6.0 was all about stabilization of the stack and I think we
> got a
>>> great headway on that. The following components/OS were in the frame of
> the
>>> discussion:
>>>      http://is.gd/H52iVe
>>>
>>> My personal take that we need to spend this release cycle working on the
>>> improvements in Bigtop itself: we got enough "technical debts" in the
>>> pipeline that have to be addressed. To name a few:
>>>    - testability/test coverage
>>>    - test framework
>>>    - package improvements
>>>    - build improvements (including performance)
>>>
>>> In order to be able to deliver a solid stack again yet improve all things
>>> Bigtop I'd like to focus on the latter, hence keeping the former at bay
> and
>>> limiting the component updates to the bugfix releases only (if
> warranted).
>>> E.g.
>>>
>>>      Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
>>>      HBase 0.94.9 (update from the Bigtop 0.6.0)
>>>      HCatalog 0.5.0 (same as 0.6.0, as well as following...)
>>>      Zookeeper 3.4.5
>>>      Pig 0.11.1
>>>      Hive 0.10.0
>>>      Sqoop 2
>>>      Oozie 3.3.2
>>>      Whirr 0.8.1
>>>      Mahout 0.7
>>>      Flume 1.3.1
>>>      Giraph 0.2.0
>>>      Hue 2.2.0
>>>      Datafu 0.0.6
>>>      Solr 4.2.1
>>>      Crunch 0.5.0
>>>      Tomcat 6.0.36
>>>      Spark 0.7.3    (it has been in the queue for a long time)
>>>
>>> Also, I'd suggest to keep the same set of OSes as last time:
>>>
>>>      CentOS/RHEL5
>>>      CentOS/RHEL6
>>>      SLES11
>>>      Ubuntu 12.04 (LTS)
>>>      Fedora 18
>>>      OpenSUSE 12.3
>>>      Ubuntu 12.10
>>>
>>> To reiterate, with a known stable version of the stack we can safely
> focus
>>> on
>>> the improvements to the framework and the overall system usability.
>>>
>>> Please jump on the discussion  Also, I have opened up the following JIRA
+1 to what Andrew and Sean said.
I would also like to add:
* I would also like to keep the door open for Apache Gora to come in.
Someone is already working on it
* Fedora 19 is out and maybe 20 will be out by the time we release 0.7.0

With regard to stability, if included all the changes listed so far seem
a little bit too much, what about dedicating a 0.6.1 to stability so
0.7.0 can include all the shiny things listed above? A 0.6.1 would
enable us to be more strict with the patches/fixes/updates being
included as well as signaling a more stable version to users.
Thanks,
Bruno
+
Konstantin Boudnik 2013-07-09, 17:40
+
Peter Linnell 2013-07-09, 20:40
+
Bruno Mahé 2013-07-10, 06:37
+
Konstantin Boudnik 2013-07-09, 17:06
+
Anatoli Fomenko 2013-07-09, 20:39
+
Konstantin Boudnik 2013-07-09, 20:54
+
Venkat Ranganathan 2013-07-09, 21:52
+
Andrew Purtell 2013-07-09, 21:58
+
Venkat Ranganathan 2013-07-09, 23:42
+
Mark Grover 2013-07-10, 00:19
+
Venkat Ranganathan 2013-07-10, 01:03
+
Jarek Jarcec Cecho 2013-07-10, 17:45
+
Andrew Purtell 2013-07-10, 00:29
+
Konstantin Boudnik 2013-07-10, 02:51
+
Sean Mackrory 2013-07-10, 04:06
+
Konstantin Boudnik 2013-07-10, 04:10
+
Bruno Mahé 2013-07-10, 06:33
+
Sean Mackrory 2013-07-10, 04:07
+
Konstantin Boudnik 2013-07-23, 16:34
+
Roman Shaposhnik 2013-08-06, 02:20
+
Konstantin Boudnik 2013-08-06, 04:17
+
Bruno Mahe 2013-07-09, 17:57
+
Konstantin Boudnik 2013-07-10, 17:52
+
Mark Grover 2013-07-10, 19:19
+
Sean Mackrory 2013-07-11, 18:24
+
Mark Grover 2013-07-12, 20:05
+
Andrew Purtell 2013-07-12, 20:12
+
Andrew Bayer 2013-07-12, 20:24
+
Andrew Purtell 2013-07-12, 20:46
+
Andrew Bayer 2013-07-12, 21:02
+
Konstantin Boudnik 2013-07-23, 16:27
+
Alan Gates 2013-07-23, 17:41
+
Andrew Purtell 2013-07-23, 18:15
+
Jarek Jarcec Cecho 2013-07-23, 18:33
+
Andrew Purtell 2013-07-23, 18:52
+
Konstantin Boudnik 2013-07-23, 18:51
+
Konstantin Boudnik 2013-07-23, 18:50
+
Konstantin Boudnik 2013-07-13, 23:55
+
Andrew Purtell 2013-07-16, 17:48
+
Konstantin Boudnik 2013-07-13, 23:50