Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Re: Question - why are there instances of org.apache.commons.lang.StringUtils and WordUtils bundled in hive?


Copy link to this message
-
Re: Question - why are there instances of org.apache.commons.lang.StringUtils and WordUtils bundled in hive?
Owen O'Malley 2013-03-25, 22:59
You're right. I was thinking there was a hive-ql jar, but there isn't.
(Note that they aren't duplicated in the source tree, just packaged up in
the jar.) I've created https://issues.apache.org/jira/browse/HIVE-4229 to
provide a jar of the ql classes without the upstream classes included.

Note that in the long stream, I think we need to simplify the jars, but
that is a bigger issue.

-- Owen
On Mon, Mar 25, 2013 at 3:23 PM, Dave Winterbourne <
[EMAIL PROTECTED]> wrote:

> We have a custom User Defined Function that extends UDF - I'll admit some
> ignorance, as I inherited this code, but UDF is a class that comes from
> hive-exec, so it doesn't seem true that hive-exec is not intended for
> external usage. That having been said, my original question is why there
> are classes from commons-lang that are simply duplicated in the code base.
> This is bad form at best, but causes class collisions and thus duplicate
> class warnings.
>
> On Mon, Mar 25, 2013 at 2:48 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:
>
> > Hive-exec isn't meant for external usage. It is the bundled jar of Hive's
> > runtime dependencies that are required for Hive's MapReduce tasks. It
> > consists of :
> >
> > hive-common
> > hive-ql
> > hive-serde
> > hive-shims
> > thrift
> > commons-lang
> > json
> > avro
> > avro-mapred
> > java-ewah
> > javolution
> > protobuf-java
> >
> > -- Owen
> >
> >
> > On Mon, Mar 25, 2013 at 11:42 AM, Dave Winterbourne <
> > [EMAIL PROTECTED]> wrote:
> >
> > > I have been working on eliminating duplicate class warnings in my maven
> > > build, and in the end discovered that there are two classes from apache
> > > commons-lang that are bundled with hive-exec:
> > >
> > > jar tf hive-0.10.0-bin//lib/hive-exec-0.10.0.jar | grep
> > > org/apache/commons/lang/
> > > org/apache/commons/lang/
> > > org/apache/commons/lang/StringUtils.class
> > > org/apache/commons/lang/WordUtils.class
> > >
> > > Why are these classes bundled with hive as opposed to just using
> > > commons-lang? If there truly is a need for custom functionality, why
> not
> > > put it in a different class to avoid this collision?
> > >
> >
>