Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - HBase NonBlocking and Async Thrift


Copy link to this message
-
Re: HBase NonBlocking and Async Thrift
Michael Segel 2012-11-21, 15:04
Are there many HBase Thrift users?

When we first looked at it 3 years ago, it wasn't commercial grade, so we decided to avoid using it.
From the look of things, it hasn't improved much and well... I would still recommend avoiding it.
All said IMHO...
On Nov 21, 2012, at 6:58 AM, Jean-Marc Spaggiari <[EMAIL PROTECTED]> wrote:

> Hi Panka,
>
> Sorry, I'm not a thrift user, but looking at your issue, I will
> recommand you to test the same think with the last version of all
> those applications.
>
> I mean, can you retry with Hadoop > 1.x and HBase 0.94.3?
>
> JM
>
> 2012/11/21, Pankaj Misra <[EMAIL PROTECTED]>:
>> Dear All,
>>
>> Requesting and looking forward for community's help on the issue, as
>> indicated in the thread below.  Thanks.
>>
>> Regards
>> Pankaj Misra
>>
>> -----Original Message-----
>> From: Pankaj Misra
>> Sent: Tuesday, November 20, 2012 12:36 PM
>> To: [EMAIL PROTECTED]
>> Subject: RE: HBase NonBlocking and Async Thrift
>>
>> Dear All,
>>
>> Requesting for help on the async thrift protocol for non-blocking streaming
>> mode, would greatly appreciate any input on the issue outlined in the thread
>> below. Putting a wait/delay defeats the purpose of using the async
>> capability. Please help.
>>
>> Thanks & Regards
>> Pankaj Misra
>>
>> -----Original Message-----
>> From: Pankaj Misra
>> Sent: Monday, November 19, 2012 5:47 PM
>> To: [EMAIL PROTECTED]
>> Subject: HBase NonBlocking and Async Thrift
>>
>> Dear All,
>>
>> I am currently using Hadoop 0.23.1 with HBase 0.94.1 in a pseudo-distributed
>> mode. I am trying to use HBase Thrift API (not using Thrift2 yet) in a
>> nonblocking and async mode to insert a bulk of records. I am sharing the set
>> of steps for everyone's information and setting the context to my problem
>>
>> Please find below the code that I am using for initializing the async
>> client
>>
>> TBinaryProtocol.Factory binProtoFactory=new TBinaryProtocol.Factory();
>>
>> TAsyncClientManager clientManager=null;
>> TNonblockingSocket nonBlockingSocket=null;
>>
>> try {
>>   clientManager=new TAsyncClientManager(); } catch (IOException e) {
>>  throw new RuntimeException(e);
>> }
>> try {
>>  nonBlockingSocket=new TNonblockingSocket(HOST_NAME,PORT_NUMBER);
>> } catch (IOException e) {
>> throw new RuntimeException(e);
>> }
>>
>>
>> And, I am initializing the client as shown below
>>
>> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
>> clientManager, nonBlockingSocket);
>>
>>
>>
>> I could see two ways of using the client, i.e. one client for all the
>> records to be inserted or separate instance of client for every record. I
>> thought since this is a non-blocking channel, it would make sense to
>> initialize 1 client for all the requests, since all the requests would be
>> streamed using a framed transport.
>>
>>
>> // 1 async client for all the requests
>> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
>> clientManager, nonBlockingSocket);
>>
>> // mutate rows called in a loop to insert multiple records, using the same
>> client client.mutateRow(table, ByteBuffer.wrap(key),
>> mutations,mutationAttributes,new HBaseInsertAsyncHandler());
>>
>>
>> But soon I found that I was wrong as I got back the following error.
>>
>> java.lang.IllegalStateException: Client is currently executing another
>> method:
>> org.apache.hadoop.hbase.thrift.generated.Hbase$AsyncClient$mutateRow_call
>>    at
>> org.apache.thrift.async.TAsyncClient.checkReady(TAsyncClient.java:78)
>>    at
>> org.apache.hadoop.hbase.thrift.generated.Hbase$AsyncClient.mutateRow(Hbase.java:2714)
>> Reading through the following JIRA educated me a bit more on this
>>
>> https://issues.apache.org/jira/browse/THRIFT-945
>>
>> So, I changed my code to initialize the client per record to be inserted.
>>
>>
>> //called both these statements for every record to be inserted in a loop
>> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,