Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - [Discuss] project chop up


Copy link to this message
-
Re: [Discuss] project chop up
Thejas Nair 2013-07-26, 17:36
+1 to the idea of making the build of core hive and other downstream
components independent.

bq.  I was under the impression that Hcat and hive-metastore was
supposed to merge up somehow.

The metastore code was never forked. Hcat was just using
hive-metastore and making the metadata available to rest of hadoop
(pig, java MR..).
A lot of the changes that were driven by hcat goals were being made in
hive-metastore. You can think of hcat as set of libraries that let pig
and java MR use hive metastore. Since hcat is closely tied to
hive-metastore, it makes sense to have them in same project.
On Fri, Jul 26, 2013 at 6:33 AM, Edward Capriolo <[EMAIL PROTECTED]> wrote:
> Also i believe hcatalog web can fall into the same designation.
>
> Question , hcatalog was initily a big hive-metastore fork. I was under the
> impression that Hcat and hive-metastore was supposed to merge up somehow.
> What is the status on that? I remember that was one of the core reasons we
> brought it in.
>
> On Friday, July 26, 2013, Edward Capriolo <[EMAIL PROTECTED]> wrote:
>> I prefer option 3 as well.
>>
>>
>> On Fri, Jul 26, 2013 at 12:52 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
>>>
>>> On Thu, Jul 25, 2013 at 9:48 PM, Edward Capriolo <[EMAIL PROTECTED]
>>wrote:
>>>
>>> > I have been developing my laptop on a duel core 2 GB Ram laptop for
> years
>>> > now. With the addition of hcatalog, hive-thrift2, and some other growth
>>> > trying to develop hive in a eclipse on this machine craws, especially
> if
>>> > 'build automatically' is turned on. As we look to add on more things
> this
>>> > is only going to get worse.
>>> >
>>> > I am also noticing issues like this:
>>> >
>>> > https://issues.apache.org/jira/browse/HIVE-4849
>>> >
>>> > What I think we should do is strip down/out optional parts of hive.
>>> >
>>> > 1) Hive Hbase
>>> >  This should really be it's own project to do this right we really
> have to
>>> > have multiple branches since hbase is not backwards compatible.
>>> >
>>> > 2) Hive Web Interface
>>> > Now really a big project but not really critical can be just as easily
> be
>>> > build separately
>>> >
>>> > 3) hive thrift 1
>>> > We have hive thrift 2 now, it is time for the sun to set on
> hivethrift1,
>>> >
>>> > 4) odbc
>>> > Not entirely convinced about this one but it is really not critical to
>>> > running hive.
>>> >
>>> > What I think we should do is create sub-projects for the above things
> or
>>> > simply move them into directories that do not build with hive. Ideally
> they
>>> > would use maven to pull dependencies.
>>> >
>>> > What does everyone think?
>>> >
>>>
>>> I agree that projects like the HBase handler and probably others as well
>>> should somehow be "downstream" projects which simply depend on the hive
>>> jars.  I see a couple alternatives for this:
>>>
>>> * Take the "module" in question to the Apache Incubator
>>> * Move the "module" in question to the Apache Extras
>>> * Breakup the projects within our own source tree
>>>
>>> I'd prefer the third option at this point.
>>>
>>> Brock
>>>
>>>
>>>
>>> Brock
>>
>>