Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> [Discuss] project chop up


Copy link to this message
-
Re: [Discuss] project chop up
Yeah I think the tar target should build the whole project.
On Jul 26, 2013 11:05 PM, "Alan Gates" <[EMAIL PROTECTED]> wrote:

> But I assume they'd still be a part of targets like package, tar, and
> binary?  Making them compile and test separately and explicitly load the
> core Hive jars from maven/ivy seems reasonable.
>
> Alan.
>
> On Jul 26, 2013, at 8:40 PM, Brock Noland wrote:
>
> > Hi,
> >
> > I think thats part of it but I'd like to decouple the downstream projects
> > even further so that the only connection is the dependency on the hive
> jars.
> >
> > Brock
> > On Jul 26, 2013 10:10 PM, "Alan Gates" <[EMAIL PROTECTED]> wrote:
> >
> >> I'm not sure how this is different from what hcat does today.  It needs
> >> Hive's jars to compile, so it's one of the last things in the compile
> step.
> >> Would moving the other modules you note to be in the same category be
> >> enough?  Did you want to also make it so that the default ant target
> >> doesn't compile those?
> >>
> >> Alan.
> >>
> >> On Jul 26, 2013, at 4:09 PM, Edward Capriolo wrote:
> >>
> >>> My mistake on saying hcat was a fork metastore. I had a brain fart for
> a
> >>> moment.
> >>>
> >>> One way we could do this is create a folder called downstream. In our
> >>> release step we can execute the downstream builds and then copy the
> files
> >>> we need back. So nothing downstream will be on the classpath of the
> main
> >>> project.
> >>>
> >>> This could help us breakup ql as well. Things like exotic file formats
> ,
> >>> and things that are pluggable like zk locking can go here. That might
> be
> >>> overkill.
> >>>
> >>> For now we can focus on building downstream and hivethrift1might be the
> >>> first thing to try to downstream.
> >>>
> >>>
> >>> On Friday, July 26, 2013, Thejas Nair <[EMAIL PROTECTED]> wrote:
> >>>> +1 to the idea of making the build of core hive and other downstream
> >>>> components independent.
> >>>>
> >>>> bq.  I was under the impression that Hcat and hive-metastore was
> >>>> supposed to merge up somehow.
> >>>>
> >>>> The metastore code was never forked. Hcat was just using
> >>>> hive-metastore and making the metadata available to rest of hadoop
> >>>> (pig, java MR..).
> >>>> A lot of the changes that were driven by hcat goals were being made in
> >>>> hive-metastore. You can think of hcat as set of libraries that let pig
> >>>> and java MR use hive metastore. Since hcat is closely tied to
> >>>> hive-metastore, it makes sense to have them in same project.
> >>>>
> >>>>
> >>>> On Fri, Jul 26, 2013 at 6:33 AM, Edward Capriolo <
> [EMAIL PROTECTED]
> >>>
> >>> wrote:
> >>>>> Also i believe hcatalog web can fall into the same designation.
> >>>>>
> >>>>> Question , hcatalog was initily a big hive-metastore fork. I was
> under
> >>> the
> >>>>> impression that Hcat and hive-metastore was supposed to merge up
> >> somehow.
> >>>>> What is the status on that? I remember that was one of the core
> reasons
> >>> we
> >>>>> brought it in.
> >>>>>
> >>>>> On Friday, July 26, 2013, Edward Capriolo <[EMAIL PROTECTED]>
> >> wrote:
> >>>>>> I prefer option 3 as well.
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Jul 26, 2013 at 12:52 AM, Brock Noland <[EMAIL PROTECTED]>
> >>> wrote:
> >>>>>>>
> >>>>>>> On Thu, Jul 25, 2013 at 9:48 PM, Edward Capriolo <
> >> [EMAIL PROTECTED]
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> I have been developing my laptop on a duel core 2 GB Ram laptop
> for
> >>>>> years
> >>>>>>>> now. With the addition of hcatalog, hive-thrift2, and some other
> >>> growth
> >>>>>>>> trying to develop hive in a eclipse on this machine craws,
> >> especially
> >>>>> if
> >>>>>>>> 'build automatically' is turned on. As we look to add on more
> things
> >>>>> this
> >>>>>>>> is only going to get worse.
> >>>>>>>>
> >>>>>>>> I am also noticing issues like this:
> >>>>>>>>
> >>>>>>>> https://issues.apache.org/jira/browse/HIVE-4849
> >>>>>>>>
> >>>>>>>> What I think we should do is strip down/out optional parts of
> hive.