Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Is Put() operation a synchronous call on server side?


Copy link to this message
-
Re: Is Put() operation a synchronous call on server side?
For a single put, yes, there is some overhead.  There is a jira to
remove the overhead: HBASE-6739, which is still open.

Thanks,
Jimmy

On Thu, Dec 6, 2012 at 10:32 AM, yun peng <[EMAIL PROTECTED]> wrote:
> Hi, Dong and Harsh, thanks for your detailed explanations. Based on Dong's
> answers, I summarise the call path a little bit.
> HTable.flushCommits()->HConnection.processBatch()->HConnectionManager#HConnectionImplementation.processBatch()->processBatchCallback()->ExecutorService.submit()
> I think there are some scheduling overheads in ExecutorService.submit().
>
> Dong, I didn't find any code related to protobuf, and my codebase is on
> HBase0.94.2. Maybe I have not use the most up-to-date version. By the way,
> I have an other and somewhat related question, but I will post in a
> seperate thread.
> Regards,
> Yun
>
> On Thu, Dec 6, 2012 at 9:06 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Hi Yun,
>>
>> Yes, a single Put call is safely synchronous in nature. A Put is
>> placed on the WAL, added to the MemStore, and then returned back as a
>> success to the client if all went well.
>>
>> A Put does not directly go to disk, and gets flushed from the MemStore
>> based on regular flushing patterns or based on manual invocations of
>> flush called on its region or its table.
>>
>> The path to follow is quite simple - A Put goes from a Client (HTable)
>> to a RegionServer (HRegionServer). You've already read the Client
>> areas, so if you read HRegionServer#put(…) method(s), which is the
>> server-end of it, you'll see the Server-RPC end of it.
>>
>> On Thu, Dec 6, 2012 at 6:32 PM, yun peng <[EMAIL PROTECTED]> wrote:
>> > Hi, since on client side HBase can immediately send Put() by turning off
>> > setAutoFlush(), I am wondering if Put() in HBase server side is executed
>> in
>> > synchronous way? Be a bit more specific, given a Put() that already
>> arrives
>> > at HRegion, will it wait (or be blocking) until all put-related
>> operations
>> > are done, such as write to WAL and write to memstore, or even flush to
>> disk
>> > (though may not on every time). Or it just triggers put-related
>> operations
>> > and immediately returns....
>> >
>> >
>> > Besides, in research of this problem, I found it not very easy to find
>> the
>> > code that perform RPC in HBase, for example, how does client-side
>> > HTable.put() invoke the server-side HRegion.put().... Can anyone points
>> to
>> > me the related code path on this regards? Thanks...
>> > regards,
>> > Yun
>>
>>
>>
>> --
>> Harsh J
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB