Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> High IPC Latency


Copy link to this message
-
Re: High IPC Latency
Hi,

We face a very similar issue since we upgraded from 0.90.3 to 0.92.
I'll test the patch very soon, and I hope to see the RS CPU become
the bottleneck as you did...

I report on this tomorrow.

Cheers,

Le 30/10/12 12:06, Yousuf Ahmad a �crit :
> Hi Lars,
>
> Thank you very much.
>
> With this fixed, I see the CPU at the regionserver become the bottleneck
> now. Today I will try 0.94 -- it has some optimizations at the regionserver
> for seeks -- to see if I can improve my benchmark numbers further.
>
> Regards,
> Yousuf
>
>
> On Tue, Oct 30, 2012 at 1:14 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
>> Whoa... Why indeed? Maybe there was a historic reason.
>> I filed HBASE-7069 to fix it.
>> -- Lars
>>
>>    ------------------------------
>> *From:* Yousuf Ahmad <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]
>> *Cc:* Ivan Brondino <[EMAIL PROTECTED]>; Ricardo Vila�a <
>> [EMAIL PROTECTED]>; lars hofhansl <[EMAIL PROTECTED]>
>> *Sent:* Monday, October 29, 2012 6:07 PM
>>
>> *Subject:* Re: High IPC Latency
>>
>> Hi guys,
>>
>> I have an update. Actually a question.
>>
>>    @Override
>>    public synchronized Object[] batch(final List<Row> actions) throws
>> InterruptedException, IOException {
>>      Object[] results = new Object[actions.size()];
>>      connection.processBatch(actions, tableName, pool, results);
>>      return results;
>>    }
>>
>> This synchronized method in HTable seems to be causing contention among
>> concurrent multigets.
>>
>> I suppose my question is: why must we have this method synchronized?
>>
>> Thanks,
>> Yousuf
>>
>>
>> On Fri, Oct 19, 2012 at 2:21 PM, Yousuf Ahmad <[EMAIL PROTECTED]> wrote:
>>
>> No coprocessors :-)
>> On Oct 19, 2012 1:21 PM, "Pamecha, Abhishek" <[EMAIL PROTECTED]> wrote:
>>
>> Also, I hope no coprocessors are in play.
>>
>> Thanks,
>> Abhishek
>>
>>
>> -----Original Message-----
>> From: Yousuf Ahmad [mailto:[EMAIL PROTECTED]]
>> Sent: Friday, October 19, 2012 10:12 AM
>> To: [EMAIL PROTECTED]
>> Cc: Ivan Brondino; Ricardo Vila�a
>> Subject: Re: High IPC Latency
>>
>> Hi Lars,
>>
>> We are following your suggestion and testing against a single region
>> server. We just ran a test against a remote region server and soon we will
>> test against a local one as well. We will get back to you soon with the
>> results.
>>
>> It will take us a couple of days to port to and test our code with 0.94.2.
>> Once we have it working, we will run some experiments and update this
>> thread.
>>
>> Unfortunately, the nature of our project is such that we cannot easily
>> translate the benchmark's workload into a program executing the equivalent
>> HBase operations directly. For this reason, I attempted to roughly
>> translate the workload in terms of HBase operations in my first email and I
>> attached a portion of the logs to be a bit more concrete.
>>
>> Your assistance is very much appreciated! Thank you! We'll keep you
>> updated.
>>
>> Best regards,
>> Yousuf
>>
>>
>> On Fri, Oct 19, 2012 at 1:25 AM, lars hofhansl <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Can you reproduce this against a single, local region server?
>>> Any chance that you can try with the just released 0.94.2?
>>>
>>>
>>> I would love to debug this. If would be a tremendous help if you had a
>>> little test program that reproduces this against a single server, so
>>> that I can see what is going on.
>>>
>>> Thanks.
>>>
>>> -- Lars
>>>
>>>
>>>
>>> ----- Original Message -----
>>> From: Yousuf Ahmad <[EMAIL PROTECTED]>
>>> To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
>>> Cc: Ivan Brondino <[EMAIL PROTECTED]>; Ricardo Vila�a <
>>> [EMAIL PROTECTED]>
>>> Sent: Thursday, October 18, 2012 12:59 PM
>>> Subject: Re: High IPC Latency
>>>
>>> Hi,
>>>
>>> Thank you for your questions guys.
>>>
>>> We are using HBase 0.92 with HDFS 1.0.1.
>>>
>>> The experiment lasts 15 minutes. The measurements stabilize in the
>>> first two minutes of the run.
>>>
>>> The data is distributed almost evenly across the regionservers so each