Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HRegionInfo was null or empty in Meta errors


+
Brandon Peskin 2013-01-31, 08:02
+
Jean-Marc Spaggiari 2013-01-31, 11:48
+
Kevin Odell 2013-01-31, 14:23
+
Brandon Peskin 2013-02-07, 23:17
Copy link to this message
-
Re: HRegionInfo was null or empty in Meta errors
HBASE-5837 is only in 0.95 and later.

Do you want HBASE-5837 to be backported ?

Thanks

On Thu, Feb 7, 2013 at 3:17 PM, Brandon Peskin <[EMAIL PROTECTED]> wrote:

> Thanks Kevin.
>
> Before I tried your advice, I tried this:
>
> scan '.META.', { FILTER =>
> org.apache.hadoop.hbase.filter.SingleColumnValueFilter.new
> (org.apache.hadoop.hbase.util.Bytes.toBytes('info'),
> org.apache.hadoop.hbase.util.Bytes.toBytes('regioninfo'),
>
> org.apache.hadoop.hbase.filter.CompareFilter::CompareOp.valueOf('NOT_EQUAL'),
> org.apache.hadoop.hbase.filter.SubstringComparator.new('algol'))}
>
> deleteall '.META.', '<row_key>'
>
>
> The problem is at some point I fat-fingereda row key and believe I hit
> HBASE-5837
>
> https://issues.apache.org/jira/browse/HBASE-5837
>
> I'm getting java.io.IOException: java.io.IOException:
> java.lang.IllegalArgumentException: No 44 in
> <2e40c841-af5b-4a5e-be0f-e06a953f05cc,1359958540596>, length=13, offset=37
> Caused my master to die, can't restart it.
>
> Is there any way around this or have I completely hosed my hbase
> installation?
>
>
> On Jan 31, 2013, at 6:23 AM, Kevin O'dell <[EMAIL PROTECTED]>
> wrote:
>
> > I am going to disagree with ignoring the error.  You will encounter
> > failures when doing other operations such as import/exports.  The first
> > thing I would do is like JM said, lets focus on the region that is not in
> > META(we at least want 0 inconsistencies).  Can you please run hbck
> -repair
> > and then run another -details and let us know if you are still seeing
> > errors?  After that, if you are still getting the NULL errors for
> > hregion:info in META. Can you please run echo "scan '.META.'" | hbase
> shell
> >> meta.out and attach the meta.out file.  I would like to take a look at
> > some of these.
> >
> > To be able to run the -repair we will want to use a different jar and
> some
> > instructions:
> >
> >
> >   1. Move the new uber jar on to the system.
> >   hbase-0.90.4-cdh3u3-patch30+3.jar
> >   2. Copy the hbase dir(/usr/lib/hbase) into /tmp/hbase dir.
> >   3. Move the hbase jar(hbase-0.90.4-cdh3u2.jar) to a .old from the tmp
> >   and replace it with the uber-hbck(hbase-0.90.4-cdh3u3-patch30+3.jar).
> >   4. break the sym links that directory
> >   5. Add the value of fs.default.name from the core-site.xml to the
> >   HBase-site.xml
> >   6. export HBASE_HOME=/tmp/hbase/ and run ./bin/hbase hbck -details 2>&1
> >   | tee details.out.
> >   7. Check the details.out and make sure you are still seeing
> >   inconsistencies
> >   8.  ./bin/hbase hbck -repair 2>&1 | tee repair.out.
> >   9. Run -details again and make sure we have 0 inconsistencies.
> >
> >
> >
> https://www.dropbox.com/s/fxotosglrrl1tq2/hbase-0.90.4-cdh3u3-patch30%2B3.jar
> > <--- new jar
> >
> > On Thu, Jan 31, 2013 at 6:48 AM, Jean-Marc Spaggiari <
> > [EMAIL PROTECTED]> wrote:
> >>
> >> Hi Brandon,
> >>
> >> I faced the same issue for "HRegionInfo was null or empty" on January
> >> 24th and Ted replied:
> >>
> >> "Encountered problems when prefetch META table:
> >>
> >> You can ignore the warning."
> >>
> >> So I think you should focus on the last one "not listed in META or
> >> deployed on any region server".
> >>
> >> Have you tried hbck to see if it can fix it?
> >>
> >> JM
> >>
> >> 2013/1/31, Brandon Peskin <[EMAIL PROTECTED]>:
> >>> hadoop 0.20.2-cdh3u2
> >>> hbase 0.90.4-cdh3u2
> >>>
> >>> On January 8th I had a network event where I lost three region servers.
> >>>
> >>> When they came back I had unassigned regions/regions not being served
> > errors
> >>> which I fixed with the hbck -fix
> >>>
> >>>
> >>>
> >>> Since then, however I have been getting an increasing number of these
> > when I
> >>> have clients trying to write to specific tables:
> >>>
> >>>
> >>> java.io.IOException: HRegionInfo was null or empty in Meta for
> >>> algol_profile_training_record,
> >>>
> >
> row=algol_profile_training_record,clientcode:49128:abce6d9f-1ee2-434a-8a82-a151b7dc183f,99999999999999
> >>
+
Kevin Odell 2013-03-04, 12:16
+
Ted 2013-03-04, 12:28
+
Ted Yu 2013-03-04, 17:46
+
Li Li 2014-05-06, 13:39