Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> High IPC Latency


+
Yousuf Ahmad 2012-10-18, 18:26
+
Pamecha, Abhishek 2012-10-18, 18:38
+
lars hofhansl 2012-10-18, 18:42
+
Yousuf Ahmad 2012-10-18, 19:59
+
lars hofhansl 2012-10-19, 05:25
+
Yousuf Ahmad 2012-10-19, 17:12
+
Pamecha, Abhishek 2012-10-19, 17:20
+
Yousuf Ahmad 2012-10-19, 18:21
+
Yousuf Ahmad 2012-10-30, 01:07
Copy link to this message
-
Re: High IPC Latency
Whoa... Why indeed? Maybe there was a historic reason.

I filed HBASE-7069 to fix it.

-- Lars

________________________________
 From: Yousuf Ahmad <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc: Ivan Brondino <[EMAIL PROTECTED]>; Ricardo Vilaça <[EMAIL PROTECTED]>; lars hofhansl <[EMAIL PROTECTED]>
Sent: Monday, October 29, 2012 6:07 PM
Subject: Re: High IPC Latency
 

Hi guys,

I have an update. Actually a question. 

  @Override
  public synchronized Object[] batch(final List<Row> actions) throws InterruptedException, IOException {
    Object[] results = new Object[actions.size()];
    connection.processBatch(actions, tableName, pool, results);
    return results;
  }

This synchronized method in HTable seems to be causing contention among concurrent multigets.

I suppose my question is: why must we have this method synchronized?

Thanks,
Yousuf

On Fri, Oct 19, 2012 at 2:21 PM, Yousuf Ahmad <[EMAIL PROTECTED]> wrote:

No coprocessors :-)
>On Oct 19, 2012 1:21 PM, "Pamecha, Abhishek" <[EMAIL PROTECTED]> wrote:
>
>Also, I hope no coprocessors are in play.
>>
>>Thanks,
>>Abhishek
>>
>>
>>-----Original Message-----
>>From: Yousuf Ahmad [mailto:[EMAIL PROTECTED]]
>>Sent: Friday, October 19, 2012 10:12 AM
>>To: [EMAIL PROTECTED]
>>Cc: Ivan Brondino; Ricardo Vilaça
>>Subject: Re: High IPC Latency
>>
>>Hi Lars,
>>
>>We are following your suggestion and testing against a single region server. We just ran a test against a remote region server and soon we will test against a local one as well. We will get back to you soon with the results.
>>
>>It will take us a couple of days to port to and test our code with 0.94.2.
>>Once we have it working, we will run some experiments and update this thread.
>>
>>Unfortunately, the nature of our project is such that we cannot easily translate the benchmark's workload into a program executing the equivalent HBase operations directly. For this reason, I attempted to roughly translate the workload in terms of HBase operations in my first email and I attached a portion of the logs to be a bit more concrete.
>>
>>Your assistance is very much appreciated! Thank you! We'll keep you updated.
>>
>>Best regards,
>>Yousuf
>>
>>
>>On Fri, Oct 19, 2012 at 1:25 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>>
>>> Can you reproduce this against a single, local region server?
>>> Any chance that you can try with the just released 0.94.2?
>>>
>>>
>>> I would love to debug this. If would be a tremendous help if you had a
>>> little test program that reproduces this against a single server, so
>>> that I can see what is going on.
>>>
>>> Thanks.
>>>
>>> -- Lars
>>>
>>>
>>>
>>> ----- Original Message -----
>>> From: Yousuf Ahmad <[EMAIL PROTECTED]>
>>> To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
>>> Cc: Ivan Brondino <[EMAIL PROTECTED]>; Ricardo Vilaça <
>>> [EMAIL PROTECTED]>
>>> Sent: Thursday, October 18, 2012 12:59 PM
>>> Subject: Re: High IPC Latency
>>>
>>> Hi,
>>>
>>> Thank you for your questions guys.
>>>
>>> We are using HBase 0.92 with HDFS 1.0.1.
>>>
>>> The experiment lasts 15 minutes. The measurements stabilize in the
>>> first two minutes of the run.
>>>
>>> The data is distributed almost evenly across the regionservers so each
>>> client hits most of them over the course of the experiment. However,
>>> for the data we have, any given multi-get or scan should touch only
>>> one or at most two regions.
>>>
>>> The client caches the locations of the regionservers, so after a
>>> couple of minutes of the experiment running, it wouldn't need to
>>> re-visit ZooKeeper, I believe. Correct me if I am wrong please.
>>>
>>> Regards,
>>> Yousuf
>>>
>>>
>>> On Thu, Oct 18, 2012 at 2:42 PM, lars hofhansl <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>> > Also, what version of HBase/HDFS is this using?
>>> >
>>> >
>>> >
>>> >
>>> > ----- Original Message -----
>>> > From: "Pamecha, Abhishek" <[EMAIL PROTECTED]>
>>> > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>> > Cc
+
Yousuf Ahmad 2012-10-30, 11:06
+
Vincent Barat 2012-11-20, 18:57
+
Ramkrishna.S.Vasudevan 2012-10-19, 04:38
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB