Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HRegionInfo was null or empty in Meta errors


+
Brandon Peskin 2013-01-31, 08:02
Copy link to this message
-
Re: HRegionInfo was null or empty in Meta errors
Hi Brandon,

I faced the same issue for "HRegionInfo was null or empty" on January
24th and Ted replied:

"Encountered problems when prefetch META table:

You can ignore the warning."

So I think you should focus on the last one "not listed in META or
deployed on any region server".

Have you tried hbck to see if it can fix it?

JM

2013/1/31, Brandon Peskin <[EMAIL PROTECTED]>:
> hadoop 0.20.2-cdh3u2
> hbase 0.90.4-cdh3u2
>
> On January 8th I had a network event where I lost three region servers.
>
> When they came back I had unassigned regions/regions not being served errors
> which I fixed with the hbck -fix
>
>
>
> Since then, however I have been getting an increasing number of these when I
> have clients trying to write to specific tables:
>
>
> java.io.IOException: HRegionInfo was null or empty in Meta for
> algol_profile_training_record,
> row=algol_profile_training_record,clientcode:49128:abce6d9f-1ee2-434a-8a82-a151b7dc183f,99999999999999
> at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:142)
> ~[hbase-0.90.4-cdh3u2.jar:na]
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
> ~[hbase-0.90.4-cdh3u2.jar:na]
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:649)
> [hbase-0.90.4-cdh3u2.jar:na]
>
>
>
> ....to the point now where I seemingly can't even write to that table.
>
> This also coincides with the following, seen in the the hbase master log:
>
> 13/01/31 02:49:55 WARN master.CatalogJanitor: REGIONINFO_QUALIFIER is empty
> in
> keyvalues={algol_tmp_client_39691_20121101053400_6,,1351783324210.b703b203d6189cad224764853cbd88f7./info:server/1351785275123/Put/vlen=32,
> algol_tmp_clientcode_39691_20121101053400_6,,1351783324210.b703b203d6189cad224764853cbd88f7./info:serverstartcode/1351785275123/Put/vlen=8}
> 13/01/31 02:49:55 WARN master.CatalogJanitor: REGIONINFO_QUALIFIER is empty
> in
> keyvalues={algol_tmp_client_39691_20121101053400_6,\x7F\xD6\xA5Pm\xF2E\x9D\x81\xB1b\xD1'\xD3\xE5\xA4,1351783324210.507bdf342aaae45644d597c272c52e02./info:server/1351785275128/Put/vlen=32,
> algol_tmp_clientcode_39691_20121101053400_6,\x7F\xD6\xA5Pm\xF2E\x9D\x81\xB1b\xD1'\xD3\xE5\xA4,1351783324210.507bdf342aaae45644d597c272c52e02./info:serverstartcode/1351785275128/Put/vlen=8}
> 13/01/31 02:49:55 WARN master.CatalogJanitor: REGIONINFO_QUALIFIER is empty
> in
> keyvalues={algol_tmp_client_34785_20121004102759_0,,1349383219155.a45774fa4c8af765df87d3373d30cfdc./info:server/1349461385375/Put/vlen=32,
> algol_tmp_overstock_34785_20121004102759_0,,1349383219155.a45774fa4c8af765df87d3373d30cfdc./info:serverstartcode/1349461385375/Put/vlen=8}
>
>
> ...and there's 58 of these I cannot fix (presumably I could with the hbck
> provided with CDH4):
>
> ERROR: Region
> hdfs://namenode:9000/hbase/algol_profile_training_record/fdaa1024d1b725b4997d2283640f0fa4
> on HDFS, but not listed in META or deployed on any region server
>
>
> I'm absolutely stumped. I've done some poking around and I can't find any
> sort of data surrounding this issue with the exception of similar symptoms
> in an exchange on this list of March this year (though the inquirer had
> different questions). Any help would be appreciated, though I suspect I will
> be told 'Upgrade to CDH4' or 'Drop and re-create the table'.
>
> Thanks in advance.
>
> --
> Brandon Peskin
> Senior Systems Administrator
> Adobe Systems
> [EMAIL PROTECTED]
>
>
>
>
>
+
Kevin Odell 2013-01-31, 14:23
+
Brandon Peskin 2013-02-07, 23:17
+
Ted Yu 2013-03-02, 15:47
+
Kevin Odell 2013-03-04, 12:16
+
Ted 2013-03-04, 12:28
+
Ted Yu 2013-03-04, 17:46
+
Li Li 2014-05-06, 13:39