Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HConnectionManager$HConnectionImplementation.locateRegionInMeta


+
Kireet 2013-05-30, 03:51
+
Anoop John 2013-05-30, 04:16
+
Kireet 2013-05-30, 13:12
+
Ted Yu 2013-05-30, 13:20
+
Kireet 2013-05-30, 14:01
+
Ted Yu 2013-05-30, 14:26
+
Himanshu Vashishtha 2013-05-30, 16:48
+
Kireet 2013-05-30, 18:28
+
Ted Yu 2013-05-30, 23:38
+
Kireet 2013-05-31, 18:58
Copy link to this message
-
Re: HConnectionManager$HConnectionImplementation.locateRegionInMeta
Indeed. That is bad.
I cannot see a clean fix immediately, but we need to look at this.

Mind filing a ticket, Kireet?

-- Lars

________________________________
 From: Kireet <[EMAIL PROTECTED]>
To: public-user-50Pas4EWwPEyzMRdD/[EMAIL PROTECTED]
Sent: Friday, May 31, 2013 11:58 AM
Subject: Re: HConnectionManager$HConnectionImplementation.locateRegionInMeta
 

Even if I initiate the call via a pooled htable, the MetaScanner seems
to use a concrete HTable instance. The constructor invoked seems to
create a java ThreadPoolExecutor. I am not 100% sure but I think as long
as nothing is submitted to the ThreadPoolExecutor it won't create any
threads. I just wanted to confirm this was the case. I do see the
connection is shared.

--Kireet

On 5/30/13 7:38 PM, Ted Yu wrote:
> HTablePool$**PooledHTable is a wrapper around HTable.
>
> Here is how HTable obtains a connection:
>
>    public HTable(Configuration conf, final byte[] tableName, final
> ExecutorService pool)
>        throws IOException {
>      this.connection = HConnectionManager.getConnection(conf);
>
> Meaning the connection is a shared one based on certain key/value pairs
> from conf.
>
> bq. So every call to batch will create a new thread?
>
> I don't think so.
>
> On Thu, May 30, 2013 at 11:28 AM, Kireet <kireet-Teh5dPVPL8nQT0dZR+[EMAIL PROTECTED]> wrote:
>
>>
>>
>> Thanks, will give it a shot. So I should download 0.94.7 (latest stable)
>> and run the patch tool on top with the backport? This is a little new to me.
>>
>> Also, I was looking at the stack below. From my reading of the code, the
>> HTable.batch() call will always cause the prefetch call to occur, which
>> will cause a new HTable object to get created. The constructor used in
>> creating a new thread pool. So every call to batch will create a new
>> thread? Or the HTable's thread pool never gets used as the pool is only
>> used for writes? I think I am missing something but just want to confirm.
>>
>> Thanks
>> Kireet
>>
>> On 5/30/13 12:48 PM, Himanshu Vashishtha wrote:
>>
>>> bq. Anoop attached backported patch in HBASE-8655. It should go into
>>>
>>> 0.94.9, the next release - current is 0.94.8
>>>
>>> In case you want it sooner, you can apply 8655 patch and test/verify it.
>>>
>>> Thanks,
>>> Himanshu
>>>
>>>
>>>
>>> On Thu, May 30, 2013 at 7:26 AM, Ted Yu <yuzhihong-**
>>> Re5JQEeQqe8AvxtiuMwx3w@public.**gmane.org<[EMAIL PROTECTED]>>
>>> wrote:
>>>
>>>   Anoop attached backported patch in HBASE-8655
>>>>
>>>> It should go into 0.94.9, the next release - current is 0.94.8
>>>>
>>>> Cheers
>>>>
>>>> On Thu, May 30, 2013 at 7:01 AM, Kireet <kireet-Teh5dPVPL8nQT0dZR+**
>>>> [EMAIL PROTECTED] <kireet-Teh5dPVPL8nQT0dZR%[EMAIL PROTECTED]>>
>>>> wrote:
>>>>
>>>>
>>>>>
>>>>> How long do backports typically take? We have to go live in a month
>>>>> ready
>>>>> or not. Thanks for the quick replies Anoop and Ted.
>>>>>
>>>>> --Kireet
>>>>>
>>>>>
>>>>> On 5/30/13 9:20 AM, Ted Yu wrote:
>>>>>
>>>>>   0.95 client is not compatible with 0.94 cluster. So you cannot use 0.95
>>>>>> client.
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>> On May 30, 2013, at 6:12 AM, Kireet <kireet-Teh5dPVPL8nQT0dZR+**
>>>>>> AlfA-XMD5yJDbdMReXY1tMh2IBg@**public.gmane.org<[EMAIL PROTECTED]><
>>>>>> kireet-Teh5dPVPL8nQT0dZR%**2BAlfA-XMD5yJDbdMReXY1tMh2IBg@**
>>>>>> public.gmane.org<kireet-Teh5dPVPL8nQT0dZR%[EMAIL PROTECTED]>
>>>>>>
>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Would there be a problem if our cluster is 0.94 and we use a 0.95
>>>>>>>
>>>>>> client?
>>>>
>>>>>
>>>>>>> I am not familiar with the HBase code base, but I did a dump of the
>>>>>>> thread that is actually running (below). It seems like it is related
>>>>>>>
>>>>>> to the
>>>>
>>>>> issue you mentioned as the running thread is doing the prefetch logic.
+
Kireet 2013-06-06, 15:25
+
Ted Yu 2013-05-30, 04:23