Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - responseTooSlow


Copy link to this message
-
Re: responseTooSlow
Mohit Anchlia 2012-12-22, 16:11
What would be the best way to check? Region Logs?

Sent from my iPhone

On Dec 22, 2012, at 6:33 AM, ramkrishna vasudevan <[EMAIL PROTECTED]> wrote:

> How many threads are doing the puts?
> Is there some flushes happening parallely?  Are there too many scans going
> on in paralle.
>
> Increasing the Xceivers could solve this problem if they are of shortage.
>
> Regards
> Ram
>
> On Sat, Dec 22, 2012 at 5:42 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
>
>> yeah
>>
>> Best Regards,
>> Tariq
>> +91-9741563634
>> https://mtariq.jux.com/
>>
>>
>> On Sat, Dec 22, 2012 at 6:57 AM, Mohit Anchlia <[EMAIL PROTECTED]
>>> wrote:
>>
>>> You mean batch multiple put?
>>>
>>> On Fri, Dec 21, 2012 at 4:16 PM, Mohammad Tariq <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>> It might be the RS which could not complete the operation in time. The
>>>> appropriate way to find out is to monitor that RS's metrics and see if
>>>> anything unusual is happening there. What type of keys are you using?
>> It
>>> is
>>>> time-series data?You might be a victim of RS hotspotting in that case
>> or
>>>> perhaps some other processes are eating up resources there. Try using
>>>> "put(List<Put> puts)" instead of "put(Put put)" and see if it makes any
>>>> difference.
>>>>
>>>> I'm afraid, I can't say anything with 100% confidence as there could be
>>> 'n'
>>>> reasons which are not traceable from here. Some of the possible reasons
>>>> could be :
>>>> hotspotting region
>>>> too much I/O wait due to Swapping
>>>> overloaded disk
>>>> slowness due to high cpu consumption
>>>>
>>>> Best Regards,
>>>> Tariq
>>>> +91-9741563634
>>>> https://mtariq.jux.com/
>>>>
>>>>
>>>> On Sat, Dec 22, 2012 at 5:23 AM, Mohit Anchlia <[EMAIL PROTECTED]
>>>>> wrote:
>>>>
>>>>> I am just doing a put. This operation generally takes 10ms but in
>> this
>>>> case
>>>>> it took more than 10sec. Nothing out of ordinary in the logs
>>>>>
>>>>> On Fri, Dec 21, 2012 at 3:26 PM, Mohammad Tariq <[EMAIL PROTECTED]>
>>>>> wrote:
>>>>>
>>>>>> what exactly is the operation your trying to do?how is your
>> network's
>>>>>> health?is swapping too high at RS side?anything odd in your RS
>> logs?
>>>>>>
>>>>>> Best Regards,
>>>>>> Tariq
>>>>>> +91-9741563634
>>>>>> https://mtariq.jux.com/
>>>>>>
>>>>>>
>>>>>> On Sat, Dec 22, 2012 at 4:36 AM, Mohit Anchlia <
>>> [EMAIL PROTECTED]
>>>>>>> wrote:
>>>>>>
>>>>>>> I looked at that link, but couldn't find anything useful. How do
>> I
>>>>> check
>>>>>> if
>>>>>>> it was client who didn't write data within that time or if it was
>>>>> region
>>>>>>> server that didn't finish operation in time.
>>>>>>>
>>>>>>> On Fri, Dec 21, 2012 at 2:54 PM, Mohammad Tariq <
>>> [EMAIL PROTECTED]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> The socket through which your client is communicating is
>> getting
>>>>> closed
>>>>>>>> before the operation could get finished. May be it is taking
>>> longer
>>>>>> than
>>>>>>>> usual or something.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Tariq
>>>>>>>> +91-9741563634
>>>>>>>> https://mtariq.jux.com/
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Dec 22, 2012 at 4:08 AM, Mohammad Tariq <
>>>> [EMAIL PROTECTED]
>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello Mohit,
>>>>>>>>>
>>>>>>>>>      You might this link<
>>>>>>>> http://hbase.apache.org/book/ops.monitoring.html>useful.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Tariq
>>>>>>>>> +91-9741563634
>>>>>>>>> https://mtariq.jux.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Dec 22, 2012 at 2:09 AM, Mohit Anchlia <
>>>>>> [EMAIL PROTECTED]
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Could someone help me understand what this really means. Is
>>> this
>>>>> the
>>>>>>>>>> network transfer taking long from client -> server or region
>>>>> server
>>>>>>>> taking
>>>>>>>>>> long time writing to the memory?
>>>>>>>>>>
>>>>>>>>>> 2012-12-21 10:54:21,980 WARN
>>> org.apache.hadoop.ipc.HBaseServer:
>>>>>>>>>> (responseTooSlow): {"processingtimems":135652,"call":"multi(