Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Thrift Hive client for CDH 4.1 HiveServer2?


Copy link to this message
-
Re: Thrift Hive client for CDH 4.1 HiveServer2?
Carl Steinbach 2013-01-28, 22:27
Hi,

The Thrift IDL file for HiveServer2 is located here:

service/if/cli_service.thrift

Here's what you need to do in order to build the HiveServer2 Perl Thrift
client using the Hive tarball from CDH4.1.x:

# Set THRIFT_HOME to point to the installation directory of Thrift 0.9.0
% export THRIFT_HOME=<...>

# Verify that THRIFT_HOME is set properly
% $THRIFT_HOME/bin/thrift -version
Thrift Version 0.9.0

# Download and unpack the hive-0.9.0-cdh4.1.2 tarball
% wget http://archive.cloudera.com/cdh4/cdh/4/hive-0.9.0-cdh4.1.2.tar.gz
% tar -xvf hive-0.9.0-cdh4.1.2.tar.gz
% cd hive-0.9.0-cdh4.1.2/src

# Generate the Perl Thrift libraries
% ant thriftif -Dthrift.home=$THRIFT_HOME -Dthrift.args="-I $THRIFT_HOME
--gen perl"

The previous command will result in the generation of the HiveServer2 Perl
library here:

service/src/gen/thrift/gen-perl/TCLIService.pm

In order to use this library you can start with the code on stackoverflow
and modify
it to reference TCLIService instead of ThriftHiveClient. Also note that the
TCLIService
Thrift service exposes a different set of functions than the
ThriftHiveClient service.
I recommend looking at the comments in TCLIService.thrift in order to
understand
the proper workflow.

Thanks.

Carl

On Sat, Jan 5, 2013 at 3:00 AM, David Morel <[EMAIL PROTECTED]> wrote:

> So that would probably be generated
> using src/service/if/cli_service.thrift instead of the older
> hive_service.thrift which i suppose is for hiveserver1.
> Compiled it, still getting errors that seem transport-related
>
> 13/01/04 23:02:22 ERROR server.TThreadPoolServer: Error occurred during
> processing of message.
> java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
>  at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:218)
> ...
>
> This is a bit tedious...
>
> D.Morel
>
>
> On Sat, Jan 5, 2013 at 10:27 AM, Jov <[EMAIL PROTECTED]> wrote:
>
>> here:
>> https://issues.apache.org/jira/browse/HIVE-2935
>> https://cwiki.apache.org/Hive/hiveserver2-thrift-api.html
>> HiveServer2 now is CDH extension.
>>
>> I think you can use find cmd to search the CDH src dir to find the
>> .thrift files.
>>
>>
>> 2013/1/5 David Morel <[EMAIL PROTECTED]>
>>
>>> On 4 Jan 2013, at 16:04, Jov wrote:
>>>
>>> they are in the src/service/if and src/metastore/if
>>>
>>> Cool. But these would be files for HiveServer, not HiveServer2 which has
>>> a different API, right? After finally generating the libs, it turns out
>>> they work fine on the old-style hive server, but produce this in
>>> hiveserver2's log: 13/01/04 20:09:11 ERROR server.TThreadPoolServer: Error
>>> occurred during processing of message. java.lang.RuntimeException:
>>> org.apache.thrift.transport.TTransportException at
>>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:218)
>>> at
>>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:170)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:662) Caused by:
>>> org.apache.thrift.transport.TTransportException at
>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at
>>> org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:124)
>>> at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:40)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:215)
>>> ... 4 more Where should I start looking (meaning I haven't a clue)? Thanks!