Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> is that possible to use Hive 0.11 with libthrift 0.7

Copy link to this message
Re: is that possible to use Hive 0.11 with libthrift 0.7
Hi, Timothy,
Thanks for your reply. Seems jarjar is a solution for me. I have a basic
question to follow:

I don't quite understand what is tag <keep> for ?

My understand is
use <include> to grab the maven dependencies I want to re-package,
then use <pattern> and <result> in <rule> to define what class to
re-package and what's the new name for re-packaged binary.
And use <keep> to indicate which class in the project will only use this
re-packaged dependencies, in my case, it is JDBC class. Am I right ?

below pom doesn't work for me, and JDBC class and other class still share
the same thrift 0.9. Thrift 0.7 is omitted.

Thanks a lot for help,

































On Thu, Oct 10, 2013 at 12:28 PM, Timothy Potter <[EMAIL PROTECTED]>wrote:

> Hi Zhang,
> I have the same issue in that I use some Cassandra client API's that
> depend on Thrift 0.7 and HCatalog 0.11 that depends on Thrift 0.9. I opted
> for using the jarjar utility to "shade" the thrift 0.9 classes. Here's what
> I added to the build.xml file for the hcatalog-pig-adapter project:
> <target name="jarjar">
>     <echo message="basedir=${basedir}"/>
>     <copy file="${basedir}/../core/build/hcatalog-core-0.11.0.jar"
> todir="${basedir}/deps"/>
>     <copy file="${basedir}/build/hcatalog-pig-adapter-0.11.0.jar"
> todir="${basedir}/deps"/>
>     <copy todir="${basedir}/deps">
>       <fileset dir="${basedir}/../../build/dist/lib"
> includes="hive-*.jar"/>
>     </copy>
>     <taskdef name="jarjar" classname="com.tonicsystems.jarjar.JarJarTask"
> classpath="lib/jarjar-1.4.jar"/>
>     <jarjar jarfile="hcat-pig-shaded.jar">
>         <zipgroupfileset dir="deps" includes="*.jar"/>
>         <rule pattern="org.apache.thrift.**" result="org.shaded.thrift.@1
> "/>
>     </jarjar>
> </target>
> I basically cooked up a deps directory that contains all the JARs that the
> hcatalog-pig-adapter depends on and then used jarjar to create a single jar
> with the thrift classes renamed at the bytecode level to org.shaded.thrift
> You can do something similar for the JDBC code.
> Cheers,
> Tim
> On Thu, Oct 10, 2013 at 1:19 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:
>> You are kinda screwed. Thrift is wire-compatible in many cases but not
>> API compatible. You can not have two applications build off two versions of
>> thrift in the same classpath without something like OSGI. To insulate the
>> class loaders from each other.
>> Your best bet is upgrading "other component" to thrift 9
>> On Thu, Oct 10, 2013 at 3:09 PM, Zhang Xiaoyu <[EMAIL PROTECTED]>wrote:
>>> Hi, all,
>>> I am writing a piece of code talking to Hive 0.11 Hive Server 2. The
>>> JDBC code depends on libthrift 0.9. However one component which depends on
>>> libthrift 0.7 and not binary compatible with libthrift 0.9.
>>> When I downgrade to 0.7, I got below NoClassDefFoundError:
>>> org/apache/thrift/scheme/StandardScheme.
>>> Any idea how to make HS2 working with thrift 0.7? Thanks.