Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - HRegionInfo was null or empty in Meta errors


+
Brandon Peskin 2013-01-31, 08:02
+
Jean-Marc Spaggiari 2013-01-31, 11:48
+
Kevin Odell 2013-01-31, 14:23
+
Brandon Peskin 2013-02-07, 23:17
+
Ted Yu 2013-03-02, 15:47
+
Kevin Odell 2013-03-04, 12:16
Copy link to this message
-
Re: HRegionInfo was null or empty in Meta errors
Ted 2013-03-04, 12:28
We should back port.

Thanks

On Mar 4, 2013, at 4:16 AM, "Kevin O'dell" <[EMAIL PROTECTED]> wrote:

> Ted,
>
>  This is the second or third time I have seen this.  I think it should
> apply fairly clean.  What do you think?
>
> On Sat, Mar 2, 2013 at 10:47 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> HBASE-5837 is only in 0.95 and later.
>>
>> Do you want HBASE-5837 to be backported ?
>>
>> Thanks
>>
>> On Thu, Feb 7, 2013 at 3:17 PM, Brandon Peskin <[EMAIL PROTECTED]> wrote:
>>
>>> Thanks Kevin.
>>>
>>> Before I tried your advice, I tried this:
>>>
>>> scan '.META.', { FILTER =>
>>> org.apache.hadoop.hbase.filter.SingleColumnValueFilter.new
>>> (org.apache.hadoop.hbase.util.Bytes.toBytes('info'),
>>> org.apache.hadoop.hbase.util.Bytes.toBytes('regioninfo'),
>> org.apache.hadoop.hbase.filter.CompareFilter::CompareOp.valueOf('NOT_EQUAL'),
>>> org.apache.hadoop.hbase.filter.SubstringComparator.new('algol'))}
>>>
>>> deleteall '.META.', '<row_key>'
>>>
>>>
>>> The problem is at some point I fat-fingereda row key and believe I hit
>>> HBASE-5837
>>>
>>> https://issues.apache.org/jira/browse/HBASE-5837
>>>
>>> I'm getting java.io.IOException: java.io.IOException:
>>> java.lang.IllegalArgumentException: No 44 in
>>> <2e40c841-af5b-4a5e-be0f-e06a953f05cc,1359958540596>, length=13,
>> offset=37
>>> Caused my master to die, can't restart it.
>>>
>>> Is there any way around this or have I completely hosed my hbase
>>> installation?
>>>
>>>
>>> On Jan 31, 2013, at 6:23 AM, Kevin O'dell <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>> I am going to disagree with ignoring the error.  You will encounter
>>>> failures when doing other operations such as import/exports.  The first
>>>> thing I would do is like JM said, lets focus on the region that is not
>> in
>>>> META(we at least want 0 inconsistencies).  Can you please run hbck
>>> -repair
>>>> and then run another -details and let us know if you are still seeing
>>>> errors?  After that, if you are still getting the NULL errors for
>>>> hregion:info in META. Can you please run echo "scan '.META.'" | hbase
>>> shell
>>>>> meta.out and attach the meta.out file.  I would like to take a look at
>>>> some of these.
>>>>
>>>> To be able to run the -repair we will want to use a different jar and
>>> some
>>>> instructions:
>>>>
>>>>
>>>>  1. Move the new uber jar on to the system.
>>>>  hbase-0.90.4-cdh3u3-patch30+3.jar
>>>>  2. Copy the hbase dir(/usr/lib/hbase) into /tmp/hbase dir.
>>>>  3. Move the hbase jar(hbase-0.90.4-cdh3u2.jar) to a .old from the tmp
>>>>  and replace it with the uber-hbck(hbase-0.90.4-cdh3u3-patch30+3.jar).
>>>>  4. break the sym links that directory
>>>>  5. Add the value of fs.default.name from the core-site.xml to the
>>>>  HBase-site.xml
>>>>  6. export HBASE_HOME=/tmp/hbase/ and run ./bin/hbase hbck -details
>> 2>&1
>>>>  | tee details.out.
>>>>  7. Check the details.out and make sure you are still seeing
>>>>  inconsistencies
>>>>  8.  ./bin/hbase hbck -repair 2>&1 | tee repair.out.
>>>>  9. Run -details again and make sure we have 0 inconsistencies.
>> https://www.dropbox.com/s/fxotosglrrl1tq2/hbase-0.90.4-cdh3u3-patch30%2B3.jar
>>>> <--- new jar
>>>>
>>>> On Thu, Jan 31, 2013 at 6:48 AM, Jean-Marc Spaggiari <
>>>> [EMAIL PROTECTED]> wrote:
>>>>>
>>>>> Hi Brandon,
>>>>>
>>>>> I faced the same issue for "HRegionInfo was null or empty" on January
>>>>> 24th and Ted replied:
>>>>>
>>>>> "Encountered problems when prefetch META table:
>>>>>
>>>>> You can ignore the warning."
>>>>>
>>>>> So I think you should focus on the last one "not listed in META or
>>>>> deployed on any region server".
>>>>>
>>>>> Have you tried hbck to see if it can fix it?
>>>>>
>>>>> JM
>>>>>
>>>>> 2013/1/31, Brandon Peskin <[EMAIL PROTECTED]>:
>>>>>> hadoop 0.20.2-cdh3u2
>>>>>> hbase 0.90.4-cdh3u2
>>>>>>
>>>>>> On January 8th I had a network event where I lost three region
>> servers.
>>>>>>
>>>>>> When they came back I had unassigned regions/regions not being served
+
Ted Yu 2013-03-04, 17:46