Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Region is not online: .META.,,1


+
Mesika, Asaf 2012-11-07, 05:28
+
Mesika, Asaf 2012-11-07, 06:08
+
Mesika, Asaf 2012-11-08, 14:31
+
Leonid Fedotov 2012-11-07, 15:43
Copy link to this message
-
Re: Region is not online: .META.,,1
Mesika, Asaf 2012-11-08, 12:23
See my 2nd mail regarding this: Didn't help.
We're still heavily investigating this, so far without any progress.

On Nov 7, 2012, at 5:43 PM, Leonid Fedotov wrote:

> The first to try is
>
> hbase hbck -repair
>
> Check here:
> http://hbase.apache.org/book/apbs04.html#d2067e11173
>
> Thank you!
>
> Sincerely,
> Leonid Fedotov
>
>
> On Nov 6, 2012, at 10:08 PM, Mesika, Asaf wrote:
>
>> After restart, running hbck results in:
>> 12/11/07 06:07:28 DEBUG client.MetaScanner: Scanning .META. starting at row= for max=2147483647 rows using org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@51b4a74b
>>
>> 12/11/07 06:07:28 INFO util.HBaseFsck: getHTableDescriptors == tableNames => []
>> Exception in thread "main" java.lang.NullPointerException
>> at org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:560)
>> at org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:346)
>> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382)
>> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120)
>>
>> HBase is version 0.94.0
>>
>>
>> On Nov 7, 2012, at 7:28 AM, Mesika, Asaf wrote:
>>
>>> Hi,
>>>
>>> I have a cluster with 10 RS and 1 Master.
>>> From some reason, any call to the HBase fails. I ran hbck and got the same failure:
>>>
>>> Tue Nov 06 13:27:07 UTC 2012, org.apache.hadoop.hbase.client.ScannerCallable@7d8a8ce2, org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException: Region is not online: .META.,,1
>>>
>>> at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:183)
>>> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:196)
>>> at org.apache.hadoop.hbase.client.MetaScanner.access$000(MetaScanner.java:48)
>>> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:126)
>>> at org.apache.hadoop.hbase.client.MetaScanner$1.connect(MetaScanner.java:123)
>>> at org.apache.hadoop.hbase.client.HConnectionManager.execute(HConnectionManager.java:359)
>>> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:123)
>>> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:99)
>>> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:77)
>>> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:61)
>>> at org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntries(HBaseFsck.java:2234)
>>> at org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:334)
>>> at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:382)
>>> at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3120)
>>>
>>>
>>> Anyone has any idea how to fix it without the obvious restart?
>>>
>>
>