Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # user - Import failure due to thrift incompatibility?


+
Lars Yencken 2013-10-08, 06:26
Copy link to this message
-
Re: Import failure due to thrift incompatibility?
Jarek Jarcec Cecho 2013-10-10, 15:54
Hi Lars,
I've seen this issue many times and I can confirm your findings - this exception is most likely being thrown because multiple incompatible thrift implementations are available on Sqoop classpath.

However Sqoop itself do not ships with thrift as we are not using thrift. Usually it's the other dependent projects that are bringing thrift as transitive dependency. Are you by any chance also using HBase? HBase is known to ship it's own thrift version that is usually not the same as the one from Hive.

Jarcec

On Tue, Oct 08, 2013 at 05:26:52PM +1100, Lars Yencken wrote:
> Hi all,
>
> I'm having trouble with what should be a standard Sqoop import job, from
> MySQL to Hive. On a new cluster, I get this exception when I do my import
> run:
>
> java.lang.NoSuchMethodError: org.apache.thrift.EncodingUtils.setBit(BIZ)B
>
> Full stack trace: https://gist.github.com/larsyencken/6879918
> Environment: Sqoop 1.4.4 / Hive 0.8.1.8 / Hadoop 1.0.3 on EMR
>
> The problem only occurs with the Hive import. If I take it out and just
> import from MySQL to HDFS, the data comes through without problems.
>
> I have a feeling the problem is to do with multiple incompatible thrift
> versions (Hive has libthrift-0.7.0.jar, Sqoop uses libthrift-0.9.0.jar).
>
> Any suggestions on getting these two to play nicely together?
>
> Best,
> Lars
+
Lars Yencken 2013-10-11, 04:56
+
Jarek Jarcec Cecho 2013-10-11, 15:36