Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Is Put() operation a synchronous call on server side?


+
yun peng 2012-12-06, 13:02
+
Harsh J 2012-12-06, 14:06
Copy link to this message
-
Re: Is Put() operation a synchronous call on server side?
Hi, Dong and Harsh, thanks for your detailed explanations. Based on Dong's
answers, I summarise the call path a little bit.
HTable.flushCommits()->HConnection.processBatch()->HConnectionManager#HConnectionImplementation.processBatch()->processBatchCallback()->ExecutorService.submit()
I think there are some scheduling overheads in ExecutorService.submit().

Dong, I didn't find any code related to protobuf, and my codebase is on
HBase0.94.2. Maybe I have not use the most up-to-date version. By the way,
I have an other and somewhat related question, but I will post in a
seperate thread.
Regards,
Yun

On Thu, Dec 6, 2012 at 9:06 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi Yun,
>
> Yes, a single Put call is safely synchronous in nature. A Put is
> placed on the WAL, added to the MemStore, and then returned back as a
> success to the client if all went well.
>
> A Put does not directly go to disk, and gets flushed from the MemStore
> based on regular flushing patterns or based on manual invocations of
> flush called on its region or its table.
>
> The path to follow is quite simple - A Put goes from a Client (HTable)
> to a RegionServer (HRegionServer). You've already read the Client
> areas, so if you read HRegionServer#put(…) method(s), which is the
> server-end of it, you'll see the Server-RPC end of it.
>
> On Thu, Dec 6, 2012 at 6:32 PM, yun peng <[EMAIL PROTECTED]> wrote:
> > Hi, since on client side HBase can immediately send Put() by turning off
> > setAutoFlush(), I am wondering if Put() in HBase server side is executed
> in
> > synchronous way? Be a bit more specific, given a Put() that already
> arrives
> > at HRegion, will it wait (or be blocking) until all put-related
> operations
> > are done, such as write to WAL and write to memstore, or even flush to
> disk
> > (though may not on every time). Or it just triggers put-related
> operations
> > and immediately returns....
> >
> >
> > Besides, in research of this problem, I found it not very easy to find
> the
> > code that perform RPC in HBase, for example, how does client-side
> > HTable.put() invoke the server-side HRegion.put().... Can anyone points
> to
> > me the related code path on this regards? Thanks...
> > regards,
> > Yun
>
>
>
> --
> Harsh J
>
+
Jimmy Xiang 2012-12-06, 18:45
+
daidong 2012-12-06, 13:43
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB