Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> [Discuss] project chop up


Copy link to this message
-
Re: [Discuss] project chop up
I have opened https://issues.apache.org/jira/browse/HIVE-5107 because I am
growing tired of how long hive's build take.

I have started playing with this by creating a simple multi-module project
and copying stuff as I go. I have ported a minimal shims and common and I
have all the tests in common almost running.

Q. This is going to be ugly hacky work for a while, I was thinking it
should be a branch but it is just going to be a mess of moves and copies
etc. Not really something you can diff etc.

Is anyone else interested in working on this as well. If so I think we can
just setup a github and I can arrange for anyone to have access to it.

Thanks,
Edward
On Wed, Aug 7, 2013 at 5:04 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> "Some of the hard part was that some of the test classes are in the wrong
> module that references classes in a later module."
>
> I think the modules will have to be able to reference each other in many
> cases. Serde and QL are tightly coupled. QL is really too large and we
> should find a way to cut that up.
>
> Part of this problem is the q.tests
>
> I think one way to handle this is to only allow unit tests inside the
> module. I imagine running all the q tests would be done in a final module
> hive-qtest. Or possibly two final modules
> hive-qtest
> hive-qtest-extra (tangential things like UDFS and input formats not core
> to hive)
>
>
> On Wed, Aug 7, 2013 at 4:49 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>
>> On Wed, Aug 7, 2013 at 12:55 PM, [EMAIL PROTECTED] <
>> [EMAIL PROTECTED]> wrote:
>>
>> > > I'd like to propose we move towards Maven.
>> >
>> > Big +1 on this. Most of the major apache projects(hadoop, hbase, avro
>> etc.)
>> > are maven based.
>> >
>>
>> A big +1 from me too. I actually took a pass at it a couple of months ago.
>> Some of the hard part was that some of the test classes are in the wrong
>> module that references classes in a later module. Obviously that prevents
>> any kind of modular build.
>>
>> As an additional plus to Maven is that Maven includes tools to correct the
>> project and module dependencies.
>>
>> -- Owen
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB