Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - HConnectionManager$HConnectionImplementation.locateRegionInMeta


Copy link to this message
-
Re: HConnectionManager$HConnectionImplementation.locateRegionInMeta
Ted Yu 2013-05-30, 23:38
HTablePool$**PooledHTable is a wrapper around HTable.

Here is how HTable obtains a connection:

  public HTable(Configuration conf, final byte[] tableName, final
ExecutorService pool)
      throws IOException {
    this.connection = HConnectionManager.getConnection(conf);

Meaning the connection is a shared one based on certain key/value pairs
from conf.

bq. So every call to batch will create a new thread?

I don't think so.

On Thu, May 30, 2013 at 11:28 AM, Kireet <[EMAIL PROTECTED]> wrote:

>
>
> Thanks, will give it a shot. So I should download 0.94.7 (latest stable)
> and run the patch tool on top with the backport? This is a little new to me.
>
> Also, I was looking at the stack below. From my reading of the code, the
> HTable.batch() call will always cause the prefetch call to occur, which
> will cause a new HTable object to get created. The constructor used in
> creating a new thread pool. So every call to batch will create a new
> thread? Or the HTable's thread pool never gets used as the pool is only
> used for writes? I think I am missing something but just want to confirm.
>
> Thanks
> Kireet
>
> On 5/30/13 12:48 PM, Himanshu Vashishtha wrote:
>
>> bq. Anoop attached backported patch in HBASE-8655. It should go into
>>
>> 0.94.9, the next release - current is 0.94.8
>>
>> In case you want it sooner, you can apply 8655 patch and test/verify it.
>>
>> Thanks,
>> Himanshu
>>
>>
>>
>> On Thu, May 30, 2013 at 7:26 AM, Ted Yu <yuzhihong-**
>> Re5JQEeQqe8AvxtiuMwx3w@public.**gmane.org<[EMAIL PROTECTED]>>
>> wrote:
>>
>>  Anoop attached backported patch in HBASE-8655
>>>
>>> It should go into 0.94.9, the next release - current is 0.94.8
>>>
>>> Cheers
>>>
>>> On Thu, May 30, 2013 at 7:01 AM, Kireet <kireet-Teh5dPVPL8nQT0dZR+**
>>> [EMAIL PROTECTED] <kireet-Teh5dPVPL8nQT0dZR%[EMAIL PROTECTED]>>
>>> wrote:
>>>
>>>
>>>>
>>>> How long do backports typically take? We have to go live in a month
>>>> ready
>>>> or not. Thanks for the quick replies Anoop and Ted.
>>>>
>>>> --Kireet
>>>>
>>>>
>>>> On 5/30/13 9:20 AM, Ted Yu wrote:
>>>>
>>>>  0.95 client is not compatible with 0.94 cluster. So you cannot use 0.95
>>>>> client.
>>>>>
>>>>> Cheers
>>>>>
>>>>> On May 30, 2013, at 6:12 AM, Kireet <kireet-Teh5dPVPL8nQT0dZR+**
>>>>> AlfA-XMD5yJDbdMReXY1tMh2IBg@**public.gmane.org<[EMAIL PROTECTED]><
>>>>> kireet-Teh5dPVPL8nQT0dZR%**2BAlfA-XMD5yJDbdMReXY1tMh2IBg@**
>>>>> public.gmane.org<kireet-Teh5dPVPL8nQT0dZR%[EMAIL PROTECTED]>
>>>>>
>>>>>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Would there be a problem if our cluster is 0.94 and we use a 0.95
>>>>>>
>>>>> client?
>>>
>>>>
>>>>>> I am not familiar with the HBase code base, but I did a dump of the
>>>>>> thread that is actually running (below). It seems like it is related
>>>>>>
>>>>> to the
>>>
>>>> issue you mentioned as the running thread is doing the prefetch logic.
>>>>>> Would pre-splitting tables help here? We are doing some performance
>>>>>>
>>>>> tests
>>>
>>>> and essentially starting from an empty instance.
>>>>>>
>>>>>> java.lang.Thread.State: WAITING (on object monitor)
>>>>>> at java.lang.Object.wait(Native Method)
>>>>>> at java.lang.Object.wait(Object.****java:503)
>>>>>> at org.apache.zookeeper.****ClientCnxn.submitRequest(**
>>>>>> ClientCnxn.java:1309)
>>>>>> - locked <0x00000000e10cf830> (a org.apache.zookeeper.**
>>>>>> ClientCnxn$Packet)
>>>>>> at org.apache.zookeeper.****ZooKeeper.exists(ZooKeeper.****java:1036)
>>>>>> at org.apache.hadoop.hbase.****zookeeper.****
>>>>>> RecoverableZooKeeper.exists(**
>>>>>> RecoverableZooKeeper.java:172)
>>>>>> at org.apache.hadoop.hbase.****zookeeper.ZKUtil.checkExists(****
>>>>>> ZKUtil.java:450)
>>>>>> at org.apache.hadoop.hbase.****zookeeper.****ZooKeeperNodeTracker.**
>>>>>> checkIfBaseNodeAvailable(****ZooKeeperNodeTracker.java:208)
>>>>>> at org.apache.hadoop.hbase.****zookeeper.RootRegionTracker.**
>>>>>> waitRootRegionLocation(****RootRegionTracker.java:77)