Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> [Discuss] project chop up

Copy link to this message
Re: [Discuss] project chop up
But I assume they'd still be a part of targets like package, tar, and binary?  Making them compile and test separately and explicitly load the core Hive jars from maven/ivy seems reasonable.


On Jul 26, 2013, at 8:40 PM, Brock Noland wrote:

> Hi,
> I think thats part of it but I'd like to decouple the downstream projects
> even further so that the only connection is the dependency on the hive jars.
> Brock
> On Jul 26, 2013 10:10 PM, "Alan Gates" <[EMAIL PROTECTED]> wrote:
>> I'm not sure how this is different from what hcat does today.  It needs
>> Hive's jars to compile, so it's one of the last things in the compile step.
>> Would moving the other modules you note to be in the same category be
>> enough?  Did you want to also make it so that the default ant target
>> doesn't compile those?
>> Alan.
>> On Jul 26, 2013, at 4:09 PM, Edward Capriolo wrote:
>>> My mistake on saying hcat was a fork metastore. I had a brain fart for a
>>> moment.
>>> One way we could do this is create a folder called downstream. In our
>>> release step we can execute the downstream builds and then copy the files
>>> we need back. So nothing downstream will be on the classpath of the main
>>> project.
>>> This could help us breakup ql as well. Things like exotic file formats ,
>>> and things that are pluggable like zk locking can go here. That might be
>>> overkill.
>>> For now we can focus on building downstream and hivethrift1might be the
>>> first thing to try to downstream.
>>> On Friday, July 26, 2013, Thejas Nair <[EMAIL PROTECTED]> wrote:
>>>> +1 to the idea of making the build of core hive and other downstream
>>>> components independent.
>>>> bq.  I was under the impression that Hcat and hive-metastore was
>>>> supposed to merge up somehow.
>>>> The metastore code was never forked. Hcat was just using
>>>> hive-metastore and making the metadata available to rest of hadoop
>>>> (pig, java MR..).
>>>> A lot of the changes that were driven by hcat goals were being made in
>>>> hive-metastore. You can think of hcat as set of libraries that let pig
>>>> and java MR use hive metastore. Since hcat is closely tied to
>>>> hive-metastore, it makes sense to have them in same project.
>>>> On Fri, Jul 26, 2013 at 6:33 AM, Edward Capriolo <[EMAIL PROTECTED]
>>> wrote:
>>>>> Also i believe hcatalog web can fall into the same designation.
>>>>> Question , hcatalog was initily a big hive-metastore fork. I was under
>>> the
>>>>> impression that Hcat and hive-metastore was supposed to merge up
>> somehow.
>>>>> What is the status on that? I remember that was one of the core reasons
>>> we
>>>>> brought it in.
>>>>> On Friday, July 26, 2013, Edward Capriolo <[EMAIL PROTECTED]>
>> wrote:
>>>>>> I prefer option 3 as well.
>>>>>> On Fri, Jul 26, 2013 at 12:52 AM, Brock Noland <[EMAIL PROTECTED]>
>>> wrote:
>>>>>>> On Thu, Jul 25, 2013 at 9:48 PM, Edward Capriolo <
>>>>>> wrote:
>>>>>>>> I have been developing my laptop on a duel core 2 GB Ram laptop for
>>>>> years
>>>>>>>> now. With the addition of hcatalog, hive-thrift2, and some other
>>> growth
>>>>>>>> trying to develop hive in a eclipse on this machine craws,
>> especially
>>>>> if
>>>>>>>> 'build automatically' is turned on. As we look to add on more things
>>>>> this
>>>>>>>> is only going to get worse.
>>>>>>>> I am also noticing issues like this:
>>>>>>>> https://issues.apache.org/jira/browse/HIVE-4849
>>>>>>>> What I think we should do is strip down/out optional parts of hive.
>>>>>>>> 1) Hive Hbase
>>>>>>>> This should really be it's own project to do this right we really
>>>>> have to
>>>>>>>> have multiple branches since hbase is not backwards compatible.
>>>>>>>> 2) Hive Web Interface
>>>>>>>> Now really a big project but not really critical can be just as
>>> easily
>>>>> be