Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase NonBlocking and Async Thrift


Copy link to this message
-
Re: HBase NonBlocking and Async Thrift
Are there many HBase Thrift users?

When we first looked at it 3 years ago, it wasn't commercial grade, so we decided to avoid using it.
From the look of things, it hasn't improved much and well... I would still recommend avoiding it.
All said IMHO...
On Nov 21, 2012, at 6:58 AM, Jean-Marc Spaggiari <[EMAIL PROTECTED]> wrote:

> Hi Panka,
>
> Sorry, I'm not a thrift user, but looking at your issue, I will
> recommand you to test the same think with the last version of all
> those applications.
>
> I mean, can you retry with Hadoop > 1.x and HBase 0.94.3?
>
> JM
>
> 2012/11/21, Pankaj Misra <[EMAIL PROTECTED]>:
>> Dear All,
>>
>> Requesting and looking forward for community's help on the issue, as
>> indicated in the thread below.  Thanks.
>>
>> Regards
>> Pankaj Misra
>>
>> -----Original Message-----
>> From: Pankaj Misra
>> Sent: Tuesday, November 20, 2012 12:36 PM
>> To: [EMAIL PROTECTED]
>> Subject: RE: HBase NonBlocking and Async Thrift
>>
>> Dear All,
>>
>> Requesting for help on the async thrift protocol for non-blocking streaming
>> mode, would greatly appreciate any input on the issue outlined in the thread
>> below. Putting a wait/delay defeats the purpose of using the async
>> capability. Please help.
>>
>> Thanks & Regards
>> Pankaj Misra
>>
>> -----Original Message-----
>> From: Pankaj Misra
>> Sent: Monday, November 19, 2012 5:47 PM
>> To: [EMAIL PROTECTED]
>> Subject: HBase NonBlocking and Async Thrift
>>
>> Dear All,
>>
>> I am currently using Hadoop 0.23.1 with HBase 0.94.1 in a pseudo-distributed
>> mode. I am trying to use HBase Thrift API (not using Thrift2 yet) in a
>> nonblocking and async mode to insert a bulk of records. I am sharing the set
>> of steps for everyone's information and setting the context to my problem
>>
>> Please find below the code that I am using for initializing the async
>> client
>>
>> TBinaryProtocol.Factory binProtoFactory=new TBinaryProtocol.Factory();
>>
>> TAsyncClientManager clientManager=null;
>> TNonblockingSocket nonBlockingSocket=null;
>>
>> try {
>>   clientManager=new TAsyncClientManager(); } catch (IOException e) {
>>  throw new RuntimeException(e);
>> }
>> try {
>>  nonBlockingSocket=new TNonblockingSocket(HOST_NAME,PORT_NUMBER);
>> } catch (IOException e) {
>> throw new RuntimeException(e);
>> }
>>
>>
>> And, I am initializing the client as shown below
>>
>> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
>> clientManager, nonBlockingSocket);
>>
>>
>>
>> I could see two ways of using the client, i.e. one client for all the
>> records to be inserted or separate instance of client for every record. I
>> thought since this is a non-blocking channel, it would make sense to
>> initialize 1 client for all the requests, since all the requests would be
>> streamed using a framed transport.
>>
>>
>> // 1 async client for all the requests
>> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
>> clientManager, nonBlockingSocket);
>>
>> // mutate rows called in a loop to insert multiple records, using the same
>> client client.mutateRow(table, ByteBuffer.wrap(key),
>> mutations,mutationAttributes,new HBaseInsertAsyncHandler());
>>
>>
>> But soon I found that I was wrong as I got back the following error.
>>
>> java.lang.IllegalStateException: Client is currently executing another
>> method:
>> org.apache.hadoop.hbase.thrift.generated.Hbase$AsyncClient$mutateRow_call
>>    at
>> org.apache.thrift.async.TAsyncClient.checkReady(TAsyncClient.java:78)
>>    at
>> org.apache.hadoop.hbase.thrift.generated.Hbase$AsyncClient.mutateRow(Hbase.java:2714)
>> Reading through the following JIRA educated me a bit more on this
>>
>> https://issues.apache.org/jira/browse/THRIFT-945
>>
>> So, I changed my code to initialize the client per record to be inserted.
>>
>>
>> //called both these statements for every record to be inserted in a loop
>> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB