Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase NonBlocking and Async Thrift


Copy link to this message
-
RE: HBase NonBlocking and Async Thrift
Thank you so much for your responses Michael and JM.

Surprisingly, HBase Definitive Guide does recommend usage of a server based client mechanism to access HBase, and mentions Avro/Thrift for best performance. I had initially tried to use Avro, which was said to be not supported in future roadmap, and I continued to use Thrift in threadpool mode. I did not face any issue in the threadpool mode, however, wanted to go a step further to make the puts more light-weight by moving towards a nonblocking mode with async capability, where I am facing the current issue.

If Thrift in nonblocking async  mode is known to have issues, I am worried/scared as to what other options I may have to access HBase using binary protocol in nonblocking mode. If we are having plans to support thrift in async mode, and is a part of the distribution, I believe there should also be a recommended way to use it as well, which I am not able to figure out and would request help for the same.

Thanks & Regards
Pankaj Misra

-----Original Message-----
From: Michael Segel [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 21, 2012 8:34 PM
To: [EMAIL PROTECTED]
Subject: Re: HBase NonBlocking and Async Thrift

Are there many HBase Thrift users?

When we first looked at it 3 years ago, it wasn't commercial grade, so we decided to avoid using it.
From the look of things, it hasn't improved much and well... I would still recommend avoiding it.
All said IMHO...
On Nov 21, 2012, at 6:58 AM, Jean-Marc Spaggiari <[EMAIL PROTECTED]> wrote:

> Hi Panka,
>
> Sorry, I'm not a thrift user, but looking at your issue, I will
> recommand you to test the same think with the last version of all
> those applications.
>
> I mean, can you retry with Hadoop > 1.x and HBase 0.94.3?
>
> JM
>
> 2012/11/21, Pankaj Misra <[EMAIL PROTECTED]>:
>> Dear All,
>>
>> Requesting and looking forward for community's help on the issue, as
>> indicated in the thread below.  Thanks.
>>
>> Regards
>> Pankaj Misra
>>
>> -----Original Message-----
>> From: Pankaj Misra
>> Sent: Tuesday, November 20, 2012 12:36 PM
>> To: [EMAIL PROTECTED]
>> Subject: RE: HBase NonBlocking and Async Thrift
>>
>> Dear All,
>>
>> Requesting for help on the async thrift protocol for non-blocking
>> streaming mode, would greatly appreciate any input on the issue
>> outlined in the thread below. Putting a wait/delay defeats the
>> purpose of using the async capability. Please help.
>>
>> Thanks & Regards
>> Pankaj Misra
>>
>> -----Original Message-----
>> From: Pankaj Misra
>> Sent: Monday, November 19, 2012 5:47 PM
>> To: [EMAIL PROTECTED]
>> Subject: HBase NonBlocking and Async Thrift
>>
>> Dear All,
>>
>> I am currently using Hadoop 0.23.1 with HBase 0.94.1 in a
>> pseudo-distributed mode. I am trying to use HBase Thrift API (not
>> using Thrift2 yet) in a nonblocking and async mode to insert a bulk
>> of records. I am sharing the set of steps for everyone's information
>> and setting the context to my problem
>>
>> Please find below the code that I am using for initializing the async
>> client
>>
>> TBinaryProtocol.Factory binProtoFactory=new
>> TBinaryProtocol.Factory();
>>
>> TAsyncClientManager clientManager=null; TNonblockingSocket
>> nonBlockingSocket=null;
>>
>> try {
>>   clientManager=new TAsyncClientManager(); } catch (IOException e) {
>> throw new RuntimeException(e); } try {  nonBlockingSocket=new
>> TNonblockingSocket(HOST_NAME,PORT_NUMBER);
>> } catch (IOException e) {
>> throw new RuntimeException(e);
>> }
>>
>>
>> And, I am initializing the client as shown below
>>
>> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
>> clientManager, nonBlockingSocket);
>>
>>
>>
>> I could see two ways of using the client, i.e. one client for all the
>> records to be inserted or separate instance of client for every record. I
>> thought since this is a non-blocking channel, it would make sense to
>> initialize 1 client for all the requests, since all the requests would be
________________________________

Neustar VP and Impetus CEO to present on ‘Innovative information services powered by Cloud and Big Data technologies’at Cloud Expo - Santa Clara, Nov 6th. http://www.impetus.com/events#2.

Check out Impetus contribution to build Luminar - a new business unit at Entravision. http://lf1.me/MS/
NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.