Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HBase NonBlocking and Async Thrift


+
Pankaj Misra 2012-11-19, 12:16
+
Pankaj Misra 2012-11-20, 07:06
+
Pankaj Misra 2012-11-21, 09:06
Copy link to this message
-
Re: HBase NonBlocking and Async Thrift
Hi Panka,

Sorry, I'm not a thrift user, but looking at your issue, I will
recommand you to test the same think with the last version of all
those applications.

I mean, can you retry with Hadoop > 1.x and HBase 0.94.3?

JM

2012/11/21, Pankaj Misra <[EMAIL PROTECTED]>:
> Dear All,
>
> Requesting and looking forward for community's help on the issue, as
> indicated in the thread below.  Thanks.
>
> Regards
> Pankaj Misra
>
> -----Original Message-----
> From: Pankaj Misra
> Sent: Tuesday, November 20, 2012 12:36 PM
> To: [EMAIL PROTECTED]
> Subject: RE: HBase NonBlocking and Async Thrift
>
> Dear All,
>
> Requesting for help on the async thrift protocol for non-blocking streaming
> mode, would greatly appreciate any input on the issue outlined in the thread
> below. Putting a wait/delay defeats the purpose of using the async
> capability. Please help.
>
> Thanks & Regards
> Pankaj Misra
>
> -----Original Message-----
> From: Pankaj Misra
> Sent: Monday, November 19, 2012 5:47 PM
> To: [EMAIL PROTECTED]
> Subject: HBase NonBlocking and Async Thrift
>
> Dear All,
>
> I am currently using Hadoop 0.23.1 with HBase 0.94.1 in a pseudo-distributed
> mode. I am trying to use HBase Thrift API (not using Thrift2 yet) in a
> nonblocking and async mode to insert a bulk of records. I am sharing the set
> of steps for everyone's information and setting the context to my problem
>
> Please find below the code that I am using for initializing the async
> client
>
> TBinaryProtocol.Factory binProtoFactory=new TBinaryProtocol.Factory();
>
> TAsyncClientManager clientManager=null;
> TNonblockingSocket nonBlockingSocket=null;
>
> try {
>    clientManager=new TAsyncClientManager(); } catch (IOException e) {
>   throw new RuntimeException(e);
> }
> try {
>   nonBlockingSocket=new TNonblockingSocket(HOST_NAME,PORT_NUMBER);
> } catch (IOException e) {
> throw new RuntimeException(e);
> }
>
>
> And, I am initializing the client as shown below
>
> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
> clientManager, nonBlockingSocket);
>
>
>
> I could see two ways of using the client, i.e. one client for all the
> records to be inserted or separate instance of client for every record. I
> thought since this is a non-blocking channel, it would make sense to
> initialize 1 client for all the requests, since all the requests would be
> streamed using a framed transport.
>
>
> // 1 async client for all the requests
> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
> clientManager, nonBlockingSocket);
>
> // mutate rows called in a loop to insert multiple records, using the same
> client client.mutateRow(table, ByteBuffer.wrap(key),
> mutations,mutationAttributes,new HBaseInsertAsyncHandler());
>
>
> But soon I found that I was wrong as I got back the following error.
>
> java.lang.IllegalStateException: Client is currently executing another
> method:
> org.apache.hadoop.hbase.thrift.generated.Hbase$AsyncClient$mutateRow_call
>     at
> org.apache.thrift.async.TAsyncClient.checkReady(TAsyncClient.java:78)
>     at
> org.apache.hadoop.hbase.thrift.generated.Hbase$AsyncClient.mutateRow(Hbase.java:2714)
> Reading through the following JIRA educated me a bit more on this
>
> https://issues.apache.org/jira/browse/THRIFT-945
>
> So, I changed my code to initialize the client per record to be inserted.
>
>
> //called both these statements for every record to be inserted in a loop
> Hbase.AsyncClient client=new Hbase.AsyncClient(binProtoFactory,
> clientManager, nonBlockingSocket); client.mutateRow(table,
> ByteBuffer.wrap(key), mutations,mutationAttributes,new
> HBaseInsertAsyncHandler());
>
> Even this failed with the following error
> 2012-11-19 17:25:15,275 WARN  [TAsyncClientManager#SelectorThread 9]
> async.TAsyncClientManager
> (TAsyncClientManager.java:startPendingMethods(177)) - Caught exception in
> TAsyncClientManager!
> java.nio.channels.ClosedChannelException
>     at
> java.nio.channels.spi.AbstractSelectableChannel.register(AbstractSelectableChannel.java:167)
+
Michael Segel 2012-11-21, 15:04
+
Pankaj Misra 2012-11-22, 12:55
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB