Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Not able to contact META (trunk version) - Master and RS on same node


Copy link to this message
-
Re: Not able to contact META (trunk version) - Master and RS on same node
It sounds like all the RS handlers were full and we're missing that
thread dump. Next time please jstack everything, alright? ;)

J-D

On Wed, Apr 3, 2013 at 11:31 PM, ramkrishna vasudevan
<[EMAIL PROTECTED]> wrote:
> Hi All
>
> During some random testing i faced this issue in Trunk version.
>
> Master and RS are on the same node and i have only one node.
>
> I created some tables, disabled and dropped it.
>
> The META was intact.
>
> Again when i tried to create the table, the Meta could not be contacted
> though in the same machine.
>
>
> On restarting the master things became fine again. I am not able to
> reproduce  this issue.  But for sometime nothing actually worked because
> the META RS became unreachable.
>
> {logs}
> 2013-04-03 17:21:21,675 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Deleting regions
> from META
> 2013-04-03 17:21:21,679 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted from META, regions: [{NAME =>
> 'TestTable,,1365024046851.e5e94a1c4adc038e6517e45bcbbab9a9.', STARTKEY =>
> '', ENDKEY => '', ENCODED => e5e94a1c4adc038e6517e45bcbbab9a9,}]
> 2013-04-03 17:21:21,683 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Archiving region
> TestTable,,1365024046851.e5e94a1c4adc038e6517e45bcbbab9a9. from FS
> 2013-04-03 17:21:21,683 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> ARCHIVING region
> hdfs://localhost:9010/hbase/.tmp/TestTable/e5e94a1c4adc038e6517e45bcbbab9a9
> ------------
> (This is where the table got deleted)
> 2013-04-03 17:21:21,705 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Deleted all region files in:
> hdfs://localhost:9010/hbase/.tmp/TestTable/e5e94a1c4adc038e6517e45bcbbab9a9
> 2013-04-03 17:21:21,706 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Table
> 'TestTable' archived!
> 2013-04-03 17:21:21,706 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Removing
> 'TestTable' descriptor.
> 2013-04-03 17:21:21,708 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Marking
> 'TestTable' as deleted.
> 2013-04-03 17:21:21,709 DEBUG
> org.apache.hadoop.hbase.master.TableLockManager: Attempt to release table
> write lock on :TestTable
> 2013-04-03 17:21:21,712 DEBUG
> org.apache.hadoop.hbase.zookeeper.lock.ZKInterProcessLockBase: Successfully
> released /hbase/table-lock/TestTable/write-master:600000000000002
> 2013-04-03 17:21:21,712 DEBUG
> org.apache.hadoop.hbase.master.TableLockManager: Released table lock on
> :TestTable
> ------
> (Again started creating the table)
>
> 2013-04-03 17:25:23,955 DEBUG
> org.apache.hadoop.hbase.master.TableLockManager: Attempt to acquire table
> write lock on :TestTable for:C_M_CREATE_TABLE
> 2013-04-03 17:25:23,961 DEBUG
> org.apache.hadoop.hbase.zookeeper.lock.ZKInterProcessLockBase: Successfully
> acquired a lock for /hbase/table-lock/TestTable/write-master:600000000000000
> 2013-04-03 17:25:23,961 DEBUG
> org.apache.hadoop.hbase.master.TableLockManager: Acquired table write lock
> on :TestTable for:C_M_CREATE_TABLE
> 2013-04-03 17:25:23,963 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Creating scanner over .META. starting at key 'TestTable,,'
> 2013-04-03 17:25:23,963 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Advancing internal scanner to startKey at 'TestTable,,'
> 2013-04-03 17:25:52,928 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> hdfs://localhost:9010/hbase/.archive/TestTable/14ffdb436fae5eec5fc0ac0a8ca89c57/info/.links-86f33ddcb5db4f7ba91f543303491908
> doesn't exist
> 2013-04-03 17:25:52,937 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> hdfs://localhost:9010/hbase/.archive/TestTable/e5e94a1c4adc038e6517e45bcbbab9a9/info/.links-be7b133a187c42b29cdd83ee555c8905
> doesn't exist
> 2013-04-03 17:25:52,940 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> hdfs://localhost:9010/hbase/.archive/TestTable/e5e94a1c4adc038e6517e45bcbbab9a9/info/.links-f30d16210ff84957a8a6800538d6884e
> doesn't exist
> 2013-04-03 17:26:05,031 DEBUG