Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - [Discuss] project chop up

Copy link to this message
Re: [Discuss] project chop up
Edward Capriolo 2013-08-16, 03:14
I have opened https://issues.apache.org/jira/browse/HIVE-5107 because I am
growing tired of how long hive's build take.

I have started playing with this by creating a simple multi-module project
and copying stuff as I go. I have ported a minimal shims and common and I
have all the tests in common almost running.

Q. This is going to be ugly hacky work for a while, I was thinking it
should be a branch but it is just going to be a mess of moves and copies
etc. Not really something you can diff etc.

Is anyone else interested in working on this as well. If so I think we can
just setup a github and I can arrange for anyone to have access to it.

On Wed, Aug 7, 2013 at 5:04 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> "Some of the hard part was that some of the test classes are in the wrong
> module that references classes in a later module."
> I think the modules will have to be able to reference each other in many
> cases. Serde and QL are tightly coupled. QL is really too large and we
> should find a way to cut that up.
> Part of this problem is the q.tests
> I think one way to handle this is to only allow unit tests inside the
> module. I imagine running all the q tests would be done in a final module
> hive-qtest. Or possibly two final modules
> hive-qtest
> hive-qtest-extra (tangential things like UDFS and input formats not core
> to hive)
> On Wed, Aug 7, 2013 at 4:49 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>> On Wed, Aug 7, 2013 at 12:55 PM, [EMAIL PROTECTED] <
>> [EMAIL PROTECTED]> wrote:
>> > > I'd like to propose we move towards Maven.
>> >
>> > Big +1 on this. Most of the major apache projects(hadoop, hbase, avro
>> etc.)
>> > are maven based.
>> >
>> A big +1 from me too. I actually took a pass at it a couple of months ago.
>> Some of the hard part was that some of the test classes are in the wrong
>> module that references classes in a later module. Obviously that prevents
>> any kind of modular build.
>> As an additional plus to Maven is that Maven includes tools to correct the
>> project and module dependencies.
>> -- Owen