Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Thrift Hive client for CDH 4.1 HiveServer2?


+
David Morel 2013-01-03, 23:16
+
Jov 2013-01-04, 15:04
+
David Morel 2013-01-05, 08:13
+
Jov 2013-01-05, 09:27
+
David Morel 2013-01-05, 11:00
Copy link to this message
-
Re: Thrift Hive client for CDH 4.1 HiveServer2?
Hi,

The Thrift IDL file for HiveServer2 is located here:

service/if/cli_service.thrift

Here's what you need to do in order to build the HiveServer2 Perl Thrift
client using the Hive tarball from CDH4.1.x:

# Set THRIFT_HOME to point to the installation directory of Thrift 0.9.0
% export THRIFT_HOME=<...>

# Verify that THRIFT_HOME is set properly
% $THRIFT_HOME/bin/thrift -version
Thrift Version 0.9.0

# Download and unpack the hive-0.9.0-cdh4.1.2 tarball
% wget http://archive.cloudera.com/cdh4/cdh/4/hive-0.9.0-cdh4.1.2.tar.gz
% tar -xvf hive-0.9.0-cdh4.1.2.tar.gz
% cd hive-0.9.0-cdh4.1.2/src

# Generate the Perl Thrift libraries
% ant thriftif -Dthrift.home=$THRIFT_HOME -Dthrift.args="-I $THRIFT_HOME
--gen perl"

The previous command will result in the generation of the HiveServer2 Perl
library here:

service/src/gen/thrift/gen-perl/TCLIService.pm

In order to use this library you can start with the code on stackoverflow
and modify
it to reference TCLIService instead of ThriftHiveClient. Also note that the
TCLIService
Thrift service exposes a different set of functions than the
ThriftHiveClient service.
I recommend looking at the comments in TCLIService.thrift in order to
understand
the proper workflow.

Thanks.

Carl

On Sat, Jan 5, 2013 at 3:00 AM, David Morel <[EMAIL PROTECTED]> wrote:

> So that would probably be generated
> using src/service/if/cli_service.thrift instead of the older
> hive_service.thrift which i suppose is for hiveserver1.
> Compiled it, still getting errors that seem transport-related
>
> 13/01/04 23:02:22 ERROR server.TThreadPoolServer: Error occurred during
> processing of message.
> java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
>  at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:218)
> ...
>
> This is a bit tedious...
>
> D.Morel
>
>
> On Sat, Jan 5, 2013 at 10:27 AM, Jov <[EMAIL PROTECTED]> wrote:
>
>> here:
>> https://issues.apache.org/jira/browse/HIVE-2935
>> https://cwiki.apache.org/Hive/hiveserver2-thrift-api.html
>> HiveServer2 now is CDH extension.
>>
>> I think you can use find cmd to search the CDH src dir to find the
>> .thrift files.
>>
>>
>> 2013/1/5 David Morel <[EMAIL PROTECTED]>
>>
>>> On 4 Jan 2013, at 16:04, Jov wrote:
>>>
>>> they are in the src/service/if and src/metastore/if
>>>
>>> Cool. But these would be files for HiveServer, not HiveServer2 which has
>>> a different API, right? After finally generating the libs, it turns out
>>> they work fine on the old-style hive server, but produce this in
>>> hiveserver2's log: 13/01/04 20:09:11 ERROR server.TThreadPoolServer: Error
>>> occurred during processing of message. java.lang.RuntimeException:
>>> org.apache.thrift.transport.TTransportException at
>>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:218)
>>> at
>>> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:170)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:662) Caused by:
>>> org.apache.thrift.transport.TTransportException at
>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at
>>> org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:124)
>>> at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:40)
>>> at
>>> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:215)
>>> ... 4 more Where should I start looking (meaning I haven't a clue)? Thanks!
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB