Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> custom Hive artifacts for Shark project


Copy link to this message
-
Re: custom Hive artifacts for Shark project
I think we plan on doing an 11.1 or just a 12.0. How does shark use hive?
Do you just include hive components from maven or does the project somehow
encorportate our build infrastructure.
On Sun, Aug 25, 2013 at 7:42 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:

> Guys,
>
> considering the absence of the input, I take it that it really doesn't
> matter
> which way the custom artifact will be published. Is it a correct
> impression?
>
> My first choice would be
>     org.apache.hive.hive-common;0.9-shark0.7
>     org.apache.hive.hive-cli;0.9-shark0.7
> artifacts.
> If this meets the objections from the community here, then I'd like to
> proceed
> with
>     org.shark-project.hive-common;0.9.0
>     org.shark-project.hive-cli;0.9.0
>
> Any of the artifacts are better be published at Maven central to make it
> readily available for development community.
>
> Thoughts?
> Regards,
>   Cos
>
> On Sat, Aug 10, 2013 at 10:08PM, Konstantin Boudnik wrote:
> > Guys,
> >
> > I am trying to help Spark/Shark community (spark-project.org and now
> > http://incubator.apache.org/projects/spark) with a predicament. Shark -
> that's
> > also known as Hive on Spark - is using some parts of Hive, ie HQL parser,
> > query optimizer, serdes, and codecs.
> >
> > In order to improve some known issues with performance and/or concurrency
> > Shark developers need to apply a couple of patches on top of the stock
> Hive:
> >    https://issues.apache.org/jira/browse/HIVE-2891
> >    https://issues.apache.org/jira/browse/HIVE-3772 (just committed to
> trunk)
> > (as per https://github.com/amplab/shark/wiki/Hive-Patches)
> >
> > The issue here is that latest Shark is working on top if Hive 0.9 (Hive
> 0.11
> > work is underway) and having developers to apply the patches and build
> > their own version of the Hive is an extra step that can be avoided.
> >
> > One way to address it is to publish Shark specific versions of Hive
> artifacts
> > that would have all needed patches applied to stock release.  This way
> > downstream projects can simply reference the version org.apache.hive with
> > version 0.9.0-shark-0.7 instead of building Hive locally every time.
> >
> > Perhaps this approach is a little overkill, so perhaps if Hive community
> is
> > willing to consider a maintenance release of Hive 0.9.1 and perhaps
> 0.11.1
> > to include fixes needed by Shark project?
> >
> > I am willing to step up and produce Hive release bits if any of the
> committers
> > here can help with publishing.
> >
> > --
> > Thanks in advance,
> >       Cos
> >
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB