Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> High IPC Latency


Copy link to this message
-
Re: High IPC Latency
Whoa... Why indeed? Maybe there was a historic reason.

I filed HBASE-7069 to fix it.

-- Lars

________________________________
 From: Yousuf Ahmad <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc: Ivan Brondino <[EMAIL PROTECTED]>; Ricardo Vilaça <[EMAIL PROTECTED]>; lars hofhansl <[EMAIL PROTECTED]>
Sent: Monday, October 29, 2012 6:07 PM
Subject: Re: High IPC Latency
 

Hi guys,

I have an update. Actually a question. 

  @Override
  public synchronized Object[] batch(final List<Row> actions) throws InterruptedException, IOException {
    Object[] results = new Object[actions.size()];
    connection.processBatch(actions, tableName, pool, results);
    return results;
  }

This synchronized method in HTable seems to be causing contention among concurrent multigets.

I suppose my question is: why must we have this method synchronized?

Thanks,
Yousuf

On Fri, Oct 19, 2012 at 2:21 PM, Yousuf Ahmad <[EMAIL PROTECTED]> wrote:

No coprocessors :-)
>On Oct 19, 2012 1:21 PM, "Pamecha, Abhishek" <[EMAIL PROTECTED]> wrote:
>
>Also, I hope no coprocessors are in play.
>>
>>Thanks,
>>Abhishek
>>
>>
>>-----Original Message-----
>>From: Yousuf Ahmad [mailto:[EMAIL PROTECTED]]
>>Sent: Friday, October 19, 2012 10:12 AM
>>To: [EMAIL PROTECTED]
>>Cc: Ivan Brondino; Ricardo Vilaça
>>Subject: Re: High IPC Latency
>>
>>Hi Lars,
>>
>>We are following your suggestion and testing against a single region server. We just ran a test against a remote region server and soon we will test against a local one as well. We will get back to you soon with the results.
>>
>>It will take us a couple of days to port to and test our code with 0.94.2.
>>Once we have it working, we will run some experiments and update this thread.
>>
>>Unfortunately, the nature of our project is such that we cannot easily translate the benchmark's workload into a program executing the equivalent HBase operations directly. For this reason, I attempted to roughly translate the workload in terms of HBase operations in my first email and I attached a portion of the logs to be a bit more concrete.
>>
>>Your assistance is very much appreciated! Thank you! We'll keep you updated.
>>
>>Best regards,
>>Yousuf
>>
>>
>>On Fri, Oct 19, 2012 at 1:25 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>>
>>> Can you reproduce this against a single, local region server?
>>> Any chance that you can try with the just released 0.94.2?
>>>
>>>
>>> I would love to debug this. If would be a tremendous help if you had a
>>> little test program that reproduces this against a single server, so
>>> that I can see what is going on.
>>>
>>> Thanks.
>>>
>>> -- Lars
>>>
>>>
>>>
>>> ----- Original Message -----
>>> From: Yousuf Ahmad <[EMAIL PROTECTED]>
>>> To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
>>> Cc: Ivan Brondino <[EMAIL PROTECTED]>; Ricardo Vilaça <
>>> [EMAIL PROTECTED]>
>>> Sent: Thursday, October 18, 2012 12:59 PM
>>> Subject: Re: High IPC Latency
>>>
>>> Hi,
>>>
>>> Thank you for your questions guys.
>>>
>>> We are using HBase 0.92 with HDFS 1.0.1.
>>>
>>> The experiment lasts 15 minutes. The measurements stabilize in the
>>> first two minutes of the run.
>>>
>>> The data is distributed almost evenly across the regionservers so each
>>> client hits most of them over the course of the experiment. However,
>>> for the data we have, any given multi-get or scan should touch only
>>> one or at most two regions.
>>>
>>> The client caches the locations of the regionservers, so after a
>>> couple of minutes of the experiment running, it wouldn't need to
>>> re-visit ZooKeeper, I believe. Correct me if I am wrong please.
>>>
>>> Regards,
>>> Yousuf
>>>
>>>
>>> On Thu, Oct 18, 2012 at 2:42 PM, lars hofhansl <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>> > Also, what version of HBase/HDFS is this using?
>>> >
>>> >
>>> >
>>> >
>>> > ----- Original Message -----
>>> > From: "Pamecha, Abhishek" <[EMAIL PROTECTED]>
>>> > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>> > Cc