Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> High IPC Latency

Copy link to this message
Re: High IPC Latency
Whoa... Why indeed? Maybe there was a historic reason.

I filed HBASE-7069 to fix it.

-- Lars

 From: Yousuf Ahmad <[EMAIL PROTECTED]>
Cc: Ivan Brondino <[EMAIL PROTECTED]>; Ricardo Vilaça <[EMAIL PROTECTED]>; lars hofhansl <[EMAIL PROTECTED]>
Sent: Monday, October 29, 2012 6:07 PM
Subject: Re: High IPC Latency

Hi guys,

I have an update. Actually a question. 

  public synchronized Object[] batch(final List<Row> actions) throws InterruptedException, IOException {
    Object[] results = new Object[actions.size()];
    connection.processBatch(actions, tableName, pool, results);
    return results;

This synchronized method in HTable seems to be causing contention among concurrent multigets.

I suppose my question is: why must we have this method synchronized?


On Fri, Oct 19, 2012 at 2:21 PM, Yousuf Ahmad <[EMAIL PROTECTED]> wrote:

No coprocessors :-)
>On Oct 19, 2012 1:21 PM, "Pamecha, Abhishek" <[EMAIL PROTECTED]> wrote:
>Also, I hope no coprocessors are in play.
>>-----Original Message-----
>>From: Yousuf Ahmad [mailto:[EMAIL PROTECTED]]
>>Sent: Friday, October 19, 2012 10:12 AM
>>Cc: Ivan Brondino; Ricardo Vilaça
>>Subject: Re: High IPC Latency
>>Hi Lars,
>>We are following your suggestion and testing against a single region server. We just ran a test against a remote region server and soon we will test against a local one as well. We will get back to you soon with the results.
>>It will take us a couple of days to port to and test our code with 0.94.2.
>>Once we have it working, we will run some experiments and update this thread.
>>Unfortunately, the nature of our project is such that we cannot easily translate the benchmark's workload into a program executing the equivalent HBase operations directly. For this reason, I attempted to roughly translate the workload in terms of HBase operations in my first email and I attached a portion of the logs to be a bit more concrete.
>>Your assistance is very much appreciated! Thank you! We'll keep you updated.
>>Best regards,
>>On Fri, Oct 19, 2012 at 1:25 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>>> Can you reproduce this against a single, local region server?
>>> Any chance that you can try with the just released 0.94.2?
>>> I would love to debug this. If would be a tremendous help if you had a
>>> little test program that reproduces this against a single server, so
>>> that I can see what is going on.
>>> Thanks.
>>> -- Lars
>>> ----- Original Message -----
>>> From: Yousuf Ahmad <[EMAIL PROTECTED]>
>>> To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
>>> Cc: Ivan Brondino <[EMAIL PROTECTED]>; Ricardo Vilaça <
>>> Sent: Thursday, October 18, 2012 12:59 PM
>>> Subject: Re: High IPC Latency
>>> Hi,
>>> Thank you for your questions guys.
>>> We are using HBase 0.92 with HDFS 1.0.1.
>>> The experiment lasts 15 minutes. The measurements stabilize in the
>>> first two minutes of the run.
>>> The data is distributed almost evenly across the regionservers so each
>>> client hits most of them over the course of the experiment. However,
>>> for the data we have, any given multi-get or scan should touch only
>>> one or at most two regions.
>>> The client caches the locations of the regionservers, so after a
>>> couple of minutes of the experiment running, it wouldn't need to
>>> re-visit ZooKeeper, I believe. Correct me if I am wrong please.
>>> Regards,
>>> Yousuf
>>> On Thu, Oct 18, 2012 at 2:42 PM, lars hofhansl <[EMAIL PROTECTED]>
>>> wrote:
>>> > Also, what version of HBase/HDFS is this using?
>>> >
>>> >
>>> >
>>> >
>>> > ----- Original Message -----
>>> > From: "Pamecha, Abhishek" <[EMAIL PROTECTED]>
>>> > Cc