Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Not able to contact META (trunk version) - Master and RS on same node


Copy link to this message
-
Re: Not able to contact META (trunk version) - Master and RS on same node
It sounds like all the RS handlers were full and we're missing that
thread dump. Next time please jstack everything, alright? ;)

J-D

On Wed, Apr 3, 2013 at 11:31 PM, ramkrishna vasudevan
<[EMAIL PROTECTED]> wrote:
> Hi All
>
> During some random testing i faced this issue in Trunk version.
>
> Master and RS are on the same node and i have only one node.
>
> I created some tables, disabled and dropped it.
>
> The META was intact.
>
> Again when i tried to create the table, the Meta could not be contacted
> though in the same machine.
>
>
> On restarting the master things became fine again. I am not able to
> reproduce  this issue.  But for sometime nothing actually worked because
> the META RS became unreachable.
>
> {logs}
> 2013-04-03 17:21:21,675 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Deleting regions
> from META
> 2013-04-03 17:21:21,679 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted from META, regions: [{NAME =>
> 'TestTable,,1365024046851.e5e94a1c4adc038e6517e45bcbbab9a9.', STARTKEY =>
> '', ENDKEY => '', ENCODED => e5e94a1c4adc038e6517e45bcbbab9a9,}]
> 2013-04-03 17:21:21,683 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Archiving region
> TestTable,,1365024046851.e5e94a1c4adc038e6517e45bcbbab9a9. from FS
> 2013-04-03 17:21:21,683 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> ARCHIVING region
> hdfs://localhost:9010/hbase/.tmp/TestTable/e5e94a1c4adc038e6517e45bcbbab9a9
> ------------
> (This is where the table got deleted)
> 2013-04-03 17:21:21,705 DEBUG org.apache.hadoop.hbase.backup.HFileArchiver:
> Deleted all region files in:
> hdfs://localhost:9010/hbase/.tmp/TestTable/e5e94a1c4adc038e6517e45bcbbab9a9
> 2013-04-03 17:21:21,706 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Table
> 'TestTable' archived!
> 2013-04-03 17:21:21,706 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Removing
> 'TestTable' descriptor.
> 2013-04-03 17:21:21,708 DEBUG
> org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Marking
> 'TestTable' as deleted.
> 2013-04-03 17:21:21,709 DEBUG
> org.apache.hadoop.hbase.master.TableLockManager: Attempt to release table
> write lock on :TestTable
> 2013-04-03 17:21:21,712 DEBUG
> org.apache.hadoop.hbase.zookeeper.lock.ZKInterProcessLockBase: Successfully
> released /hbase/table-lock/TestTable/write-master:600000000000002
> 2013-04-03 17:21:21,712 DEBUG
> org.apache.hadoop.hbase.master.TableLockManager: Released table lock on
> :TestTable
> ------
> (Again started creating the table)
>
> 2013-04-03 17:25:23,955 DEBUG
> org.apache.hadoop.hbase.master.TableLockManager: Attempt to acquire table
> write lock on :TestTable for:C_M_CREATE_TABLE
> 2013-04-03 17:25:23,961 DEBUG
> org.apache.hadoop.hbase.zookeeper.lock.ZKInterProcessLockBase: Successfully
> acquired a lock for /hbase/table-lock/TestTable/write-master:600000000000000
> 2013-04-03 17:25:23,961 DEBUG
> org.apache.hadoop.hbase.master.TableLockManager: Acquired table write lock
> on :TestTable for:C_M_CREATE_TABLE
> 2013-04-03 17:25:23,963 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Creating scanner over .META. starting at key 'TestTable,,'
> 2013-04-03 17:25:23,963 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
> Advancing internal scanner to startKey at 'TestTable,,'
> 2013-04-03 17:25:52,928 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> hdfs://localhost:9010/hbase/.archive/TestTable/14ffdb436fae5eec5fc0ac0a8ca89c57/info/.links-86f33ddcb5db4f7ba91f543303491908
> doesn't exist
> 2013-04-03 17:25:52,937 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> hdfs://localhost:9010/hbase/.archive/TestTable/e5e94a1c4adc038e6517e45bcbbab9a9/info/.links-be7b133a187c42b29cdd83ee555c8905
> doesn't exist
> 2013-04-03 17:25:52,940 DEBUG org.apache.hadoop.hbase.util.FSUtils:
> hdfs://localhost:9010/hbase/.archive/TestTable/e5e94a1c4adc038e6517e45bcbbab9a9/info/.links-f30d16210ff84957a8a6800538d6884e
> doesn't exist
> 2013-04-03 17:26:05,031 DEBUG
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB