Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Maven unit test question

Copy link to this message
Maven unit test question
Alan Gates 2013-12-09, 23:14
I was attempting to write unit tests for changes I'm making to HiveMetaStoreClient as part of the ACID transaction work (see https://issues.apache.org/jira/browse/HIVE-5843).  When I added the tests and attempted to run them using
mvn tests -Dtest=TestHiveMetaStoreClient -Phadoop-1

it failed with:

java.lang.NoClassDefFoundError: org/apache/hadoop/hive/thrift/TUGIContainingTransport$Factory

This class is contained in the hive-shims jar.  The error surprised me because according to metastore/pom.xml, hive-shims is a dependency of hive-metastore.  When I ran maven with -X to get debug information, I found that in the classpath it was including /Users/gates/git/apache/hive/shims/assembly/target/classes.  I'm guessing that rather than use the shims jar (which has been built by this time) it's trying to use the compiled classes, but failing in this case because the shims jar is actually constructed not by directly conglomerating a set of class files but by picking and choosing from several shim jar versions and then constructing a single jar.  But I could not figure out how to communicate to maven that is should use the already built shims jar rather than the classes.  To test my theory I took the shims jar and unpacked in the path maven was looking in, and sure enough my tests ran once I did that.

The existing unit test TestMetastoreExpr in ql seems to have the same issue.  I tried to use it as a model, but when I ran it it failed with the same error, and unpacking the jar resolved it in the same way.

Am I doing something wrong, or is there a change needed in the pom.xml to get it to look in the jar instead of the .class files for shims dependencies?

NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.