Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> Review Request 18927: DRILL-398: Fix build/packaging issues with storage-hive jars

Copy link to this message
Review Request 18927: DRILL-398: Fix build/packaging issues with storage-hive jars

This is an automatically generated e-mail. To reply, visit:

Review request for drill.
Bugs: DRILL-398
Repository: drill-git

1. Remove hive-exec dependency from root pom.xml and move hive-hbase dependency to contrib/storage-hive/pom.xml. The idea is to download all required hive dependencies through storage-hive artifact. Storage-hive.jar is the shaded version of hive-exec uber jar.
2. In distribution/pom.xml exclude hive-exec jar from storage-hive dependency. Storage-hive jar already contains the shaded hive-exec.
3. In contrib/storage-hive/pom.xml add a target to unpack the classes in storage-hive.jar into contrib/storage-hive/target/classes directory. Unpacking is needed because when you run 'mvn test' from root directory, sub modules dependency classes from dir /target/classes are added to CLASSPATH instead of packaged jar.

Ex. running a test in sqlparser module which depends on contrib/storage-hive module. Instead of adding storage-hive.jar to CLASSPATH, maven adds contrib/storage-hive/target/classes. From the web serach, this looks like by design in Maven. If we don't unpack the jar, we get class not found errors for storage-hive.jar classes. This is a problem for storage-hive.jar and not for other modules, because storage-hive is generated from shading the hive-exec jar instead of source files. As there are no source files contrib/storage-hive/target/classes is expected to empty.

  contrib/storage-hive/pom.xml 237d6d7
  distribution/pom.xml ffbac0f
  pom.xml 1bb58c0

Diff: https://reviews.apache.org/r/18927/diff/

After mvn install, made sure distribution tar doesn't contain hive-exec in lib
Ran individual tests with "mvn test -Dtest=..." from project root

Venki Korukanti