Hey all,

I'm running hbase 0.90.4, and somehow the regions for one of my tables have
become unlinked from the table itself.  The Master UI shows no regions for
the table, and attempting to call Table.getRegionsInfo() returns nothing.
 However, I can see the regions in hdfs in their proper location and doing
a scan of the table via the hbase shell does return results.  I also get
results if I do a Get for a specific rowkey.

How can I get these regions back associated with the table, and any idea
why this may have happened?  I tried grepping the hmaster logs but didn't
see anything relevant.

Thanks!

Bryan

Try doing an hbase hbck to see if it reports inconsistency. And do an
hbase hbck -fix to see if it fixes it for you. See
http://hbase.apache.org/book.html#hbck.in.depth

Note that since 0.90.4 is old, some of the documented options won't be
available ... but hbase hbck -fix will be available.
--S

On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
<[EMAIL PROTECTED]> wrote:
> Hey all,
>
> I'm running hbase 0.90.4, and somehow the regions for one of my tables have
> become unlinked from the table itself.  The Master UI shows no regions for
> the table, and attempting to call Table.getRegionsInfo() returns nothing.
>  However, I can see the regions in hdfs in their proper location and doing
> a scan of the table via the hbase shell does return results.  I also get
> results if I do a Get for a specific rowkey.
>
> How can I get these regions back associated with the table, and any idea
> why this may have happened?  I tried grepping the hmaster logs but didn't
> see anything relevant.
>
> Thanks!
>
> Bryan

Thanks for the response.

hbck consistently shows 7 inconsistencies.  I tried -fix (a few times) and
it didn't seem to fix it.  I tried -details and it does seem to print out
more, but nothing super usable from what I can tell.

On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:

> Try doing an hbase hbck to see if it reports inconsistency. And do an
> hbase hbck -fix to see if it fixes it for you. See
> http://hbase.apache.org/book.html#hbck.in.depth
>
> Note that since 0.90.4 is old, some of the documented options won't be
> available ... but hbase hbck -fix will be available.
> --S
>
> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
> <[EMAIL PROTECTED]> wrote:
> > Hey all,
> >
> > I'm running hbase 0.90.4, and somehow the regions for one of my tables
> have
> > become unlinked from the table itself.  The Master UI shows no regions
> for
> > the table, and attempting to call Table.getRegionsInfo() returns nothing.
> >  However, I can see the regions in hdfs in their proper location and
> doing
> > a scan of the table via the hbase shell does return results.  I also get
> > results if I do a Get for a specific rowkey.
> >
> > How can I get these regions back associated with the table, and any idea
> > why this may have happened?  I tried grepping the hmaster logs but didn't
> > see anything relevant.
> >
> > Thanks!
> >
> > Bryan
>

Are you using apache hbase 0.90.4 ... or the one from CDH3?
Check what other hbck options you have (do you have -fixMeta?) on the
version you are on.

What the uber-hbck (part of hbase-0.90.6 and later releases) does is
"automate" this meta table surgery to fix the issue.

I will let folks who have tried other repair options respond to you.
--S

On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
<[EMAIL PROTECTED]> wrote:
> Thanks for the response.
>
> hbck consistently shows 7 inconsistencies.  I tried -fix (a few times) and
> it didn't seem to fix it.  I tried -details and it does seem to print out
> more, but nothing super usable from what I can tell.
>
> On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>
>> Try doing an hbase hbck to see if it reports inconsistency. And do an
>> hbase hbck -fix to see if it fixes it for you. See
>> http://hbase.apache.org/book.html#hbck.in.depth
>>
>> Note that since 0.90.4 is old, some of the documented options won't be
>> available ... but hbase hbck -fix will be available.
>> --S
>>
>> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
>> <[EMAIL PROTECTED]> wrote:
>> > Hey all,
>> >
>> > I'm running hbase 0.90.4, and somehow the regions for one of my tables
>> have
>> > become unlinked from the table itself.  The Master UI shows no regions
>> for
>> > the table, and attempting to call Table.getRegionsInfo() returns nothing.
>> >  However, I can see the regions in hdfs in their proper location and
>> doing
>> > a scan of the table via the hbase shell does return results.  I also get
>> > results if I do a Get for a specific rowkey.
>> >
>> > How can I get these regions back associated with the table, and any idea
>> > why this may have happened?  I tried grepping the hmaster logs but didn't
>> > see anything relevant.
>> >
>> > Thanks!
>> >
>> > Bryan
>>

I'm on cdh3u2 to be precise.  I don't have the fixMeta option, or any other
special fix options aside from the basic -fix.

I'll take a look at what 0.90.6 version does when it fixes meta.  Maybe I
can just follow what it does there.

Thanks again!

On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:

> Are you using apache hbase 0.90.4 ... or the one from CDH3?
> Check what other hbck options you have (do you have -fixMeta?) on the
> version you are on.
>
> What the uber-hbck (part of hbase-0.90.6 and later releases) does is
> "automate" this meta table surgery to fix the issue.
>
> I will let folks who have tried other repair options respond to you.
> --S
>
> On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]> wrote:
> > Thanks for the response.
> >
> > hbck consistently shows 7 inconsistencies.  I tried -fix (a few times)
> and
> > it didn't seem to fix it.  I tried -details and it does seem to print out
> > more, but nothing super usable from what I can tell.
> >
> > On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
> >
> >> Try doing an hbase hbck to see if it reports inconsistency. And do an
> >> hbase hbck -fix to see if it fixes it for you. See
> >> http://hbase.apache.org/book.html#hbck.in.depth
> >>
> >> Note that since 0.90.4 is old, some of the documented options won't be
> >> available ... but hbase hbck -fix will be available.
> >> --S
> >>
> >> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
> >> <[EMAIL PROTECTED]> wrote:
> >> > Hey all,
> >> >
> >> > I'm running hbase 0.90.4, and somehow the regions for one of my tables
> >> have
> >> > become unlinked from the table itself.  The Master UI shows no regions
> >> for
> >> > the table, and attempting to call Table.getRegionsInfo() returns
> nothing.
> >> >  However, I can see the regions in hdfs in their proper location and
> >> doing
> >> > a scan of the table via the hbase shell does return results.  I also
> get
> >> > results if I do a Get for a specific rowkey.
> >> >
> >> > How can I get these regions back associated with the table, and any
> idea
> >> > why this may have happened?  I tried grepping the hmaster logs but
> didn't
> >> > see anything relevant.
> >> >
> >> > Thanks!
> >> >
> >> > Bryan
> >>
>

Bryan,
There is a script attached to
https://issues.apache.org/jira/browse/HBASE-4232 , I have used it a couple
of times to add region on filesystem but not in .META.

A sample run goes like this (note you need to provide path to regioninfo):

${HBASE_HOME}/bin/hbase org.jruby.Main addregion.rb
/hbase/mytable/974303001c9100070f54dbc39ebafa59/.regioninfo

On Mon, Jul 2, 2012 at 4:44 PM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> I'm on cdh3u2 to be precise.  I don't have the fixMeta option, or any other
> special fix options aside from the basic -fix.
>
> I'll take a look at what 0.90.6 version does when it fixes meta.  Maybe I
> can just follow what it does there.
>
> Thanks again!
>
> On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>
> > Are you using apache hbase 0.90.4 ... or the one from CDH3?
> > Check what other hbck options you have (do you have -fixMeta?) on the
> > version you are on.
> >
> > What the uber-hbck (part of hbase-0.90.6 and later releases) does is
> > "automate" this meta table surgery to fix the issue.
> >
> > I will let folks who have tried other repair options respond to you.
> > --S
> >
> > On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
> > <[EMAIL PROTECTED]> wrote:
> > > Thanks for the response.
> > >
> > > hbck consistently shows 7 inconsistencies.  I tried -fix (a few times)
> > and
> > > it didn't seem to fix it.  I tried -details and it does seem to print
> out
> > > more, but nothing super usable from what I can tell.
> > >
> > > On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]>
> wrote:
> > >
> > >> Try doing an hbase hbck to see if it reports inconsistency. And do an
> > >> hbase hbck -fix to see if it fixes it for you. See
> > >> http://hbase.apache.org/book.html#hbck.in.depth
> > >>
> > >> Note that since 0.90.4 is old, some of the documented options won't be
> > >> available ... but hbase hbck -fix will be available.
> > >> --S
> > >>
> > >> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
> > >> <[EMAIL PROTECTED]> wrote:
> > >> > Hey all,
> > >> >
> > >> > I'm running hbase 0.90.4, and somehow the regions for one of my
> tables
> > >> have
> > >> > become unlinked from the table itself.  The Master UI shows no
> regions
> > >> for
> > >> > the table, and attempting to call Table.getRegionsInfo() returns
> > nothing.
> > >> >  However, I can see the regions in hdfs in their proper location and
> > >> doing
> > >> > a scan of the table via the hbase shell does return results.  I also
> > get
> > >> > results if I do a Get for a specific rowkey.
> > >> >
> > >> > How can I get these regions back associated with the table, and any
> > idea
> > >> > why this may have happened?  I tried grepping the hmaster logs but
> > didn't
> > >> > see anything relevant.
> > >> >
> > >> > Thanks!
> > >> >
> > >> > Bryan
> > >>
> >
>

I was about to propose that next ... you could just try back porting
that uber hbck (or the relevant portions) and run the patched ruby
script ... I've done this for like online meta table merges ... but
not with hbck.

So - yes, that might be your solve. Good luck!
--S

On Mon, Jul 2, 2012 at 4:44 PM, Bryan Beaudreault
<[EMAIL PROTECTED]> wrote:
> I'm on cdh3u2 to be precise.  I don't have the fixMeta option, or any other
> special fix options aside from the basic -fix.
>
> I'll take a look at what 0.90.6 version does when it fixes meta.  Maybe I
> can just follow what it does there.
>
> Thanks again!
>
> On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>
>> Are you using apache hbase 0.90.4 ... or the one from CDH3?
>> Check what other hbck options you have (do you have -fixMeta?) on the
>> version you are on.
>>
>> What the uber-hbck (part of hbase-0.90.6 and later releases) does is
>> "automate" this meta table surgery to fix the issue.
>>
>> I will let folks who have tried other repair options respond to you.
>> --S
>>
>> On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
>> <[EMAIL PROTECTED]> wrote:
>> > Thanks for the response.
>> >
>> > hbck consistently shows 7 inconsistencies.  I tried -fix (a few times)
>> and
>> > it didn't seem to fix it.  I tried -details and it does seem to print out
>> > more, but nothing super usable from what I can tell.
>> >
>> > On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>> >
>> >> Try doing an hbase hbck to see if it reports inconsistency. And do an
>> >> hbase hbck -fix to see if it fixes it for you. See
>> >> http://hbase.apache.org/book.html#hbck.in.depth
>> >>
>> >> Note that since 0.90.4 is old, some of the documented options won't be
>> >> available ... but hbase hbck -fix will be available.
>> >> --S
>> >>
>> >> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
>> >> <[EMAIL PROTECTED]> wrote:
>> >> > Hey all,
>> >> >
>> >> > I'm running hbase 0.90.4, and somehow the regions for one of my tables
>> >> have
>> >> > become unlinked from the table itself.  The Master UI shows no regions
>> >> for
>> >> > the table, and attempting to call Table.getRegionsInfo() returns
>> nothing.
>> >> >  However, I can see the regions in hdfs in their proper location and
>> >> doing
>> >> > a scan of the table via the hbase shell does return results.  I also
>> get
>> >> > results if I do a Get for a specific rowkey.
>> >> >
>> >> > How can I get these regions back associated with the table, and any
>> idea
>> >> > why this may have happened?  I tried grepping the hmaster logs but
>> didn't
>> >> > see anything relevant.
>> >> >
>> >> > Thanks!
>> >> >
>> >> > Bryan
>> >>
>>

Can you please attach the output of hbck -details and the output of hadoop
fs -lsr /hbase(you can send directly if you do not want to share that)?
 There should not be any problem with using the uber-hbck to resolve this
from the .90.6 version.

On Mon, Jul 2, 2012 at 8:38 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:

> I was about to propose that next ... you could just try back porting
> that uber hbck (or the relevant portions) and run the patched ruby
> script ... I've done this for like online meta table merges ... but
> not with hbck.
>
> So - yes, that might be your solve. Good luck!
> --S
>
> On Mon, Jul 2, 2012 at 4:44 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]> wrote:
> > I'm on cdh3u2 to be precise.  I don't have the fixMeta option, or any
> other
> > special fix options aside from the basic -fix.
> >
> > I'll take a look at what 0.90.6 version does when it fixes meta.  Maybe I
> > can just follow what it does there.
> >
> > Thanks again!
> >
> > On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
> >
> >> Are you using apache hbase 0.90.4 ... or the one from CDH3?
> >> Check what other hbck options you have (do you have -fixMeta?) on the
> >> version you are on.
> >>
> >> What the uber-hbck (part of hbase-0.90.6 and later releases) does is
> >> "automate" this meta table surgery to fix the issue.
> >>
> >> I will let folks who have tried other repair options respond to you.
> >> --S
> >>
> >> On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
> >> <[EMAIL PROTECTED]> wrote:
> >> > Thanks for the response.
> >> >
> >> > hbck consistently shows 7 inconsistencies.  I tried -fix (a few times)
> >> and
> >> > it didn't seem to fix it.  I tried -details and it does seem to print
> out
> >> > more, but nothing super usable from what I can tell.
> >> >
> >> > On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]>
> wrote:
> >> >
> >> >> Try doing an hbase hbck to see if it reports inconsistency. And do an
> >> >> hbase hbck -fix to see if it fixes it for you. See
> >> >> http://hbase.apache.org/book.html#hbck.in.depth
> >> >>
> >> >> Note that since 0.90.4 is old, some of the documented options won't
> be
> >> >> available ... but hbase hbck -fix will be available.
> >> >> --S
> >> >>
> >> >> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
> >> >> <[EMAIL PROTECTED]> wrote:
> >> >> > Hey all,
> >> >> >
> >> >> > I'm running hbase 0.90.4, and somehow the regions for one of my
> tables
> >> >> have
> >> >> > become unlinked from the table itself.  The Master UI shows no
> regions
> >> >> for
> >> >> > the table, and attempting to call Table.getRegionsInfo() returns
> >> nothing.
> >> >> >  However, I can see the regions in hdfs in their proper location
> and
> >> >> doing
> >> >> > a scan of the table via the hbase shell does return results.  I
> also
> >> get
> >> >> > results if I do a Get for a specific rowkey.
> >> >> >
> >> >> > How can I get these regions back associated with the table, and any
> >> idea
> >> >> > why this may have happened?  I tried grepping the hmaster logs but
> >> didn't
> >> >> > see anything relevant.
> >> >> >
> >> >> > Thanks!
> >> >> >
> >> >> > Bryan
> >> >>
> >>
>

--
Kevin O'Dell
Customer Operations Engineer, Cloudera

Bryan,

  I forgot to add the link to the new code to grab the uber-hbck
http://archive.cloudera.com/cdh/3/hbase-0.90.6-cdh3u4.tar.gz

On Mon, Jul 9, 2012 at 12:53 PM, Kevin O'dell <[EMAIL PROTECTED]>wrote:

> Can you please attach the output of hbck -details and the output of hadoop
> fs -lsr /hbase(you can send directly if you do not want to share that)?
>  There should not be any problem with using the uber-hbck to resolve this
> from the .90.6 version.
>
>
> On Mon, Jul 2, 2012 at 8:38 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>
>> I was about to propose that next ... you could just try back porting
>> that uber hbck (or the relevant portions) and run the patched ruby
>> script ... I've done this for like online meta table merges ... but
>> not with hbck.
>>
>> So - yes, that might be your solve. Good luck!
>> --S
>>
>> On Mon, Jul 2, 2012 at 4:44 PM, Bryan Beaudreault
>> <[EMAIL PROTECTED]> wrote:
>> > I'm on cdh3u2 to be precise.  I don't have the fixMeta option, or any
>> other
>> > special fix options aside from the basic -fix.
>> >
>> > I'll take a look at what 0.90.6 version does when it fixes meta.  Maybe
>> I
>> > can just follow what it does there.
>> >
>> > Thanks again!
>> >
>> > On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> Are you using apache hbase 0.90.4 ... or the one from CDH3?
>> >> Check what other hbck options you have (do you have -fixMeta?) on the
>> >> version you are on.
>> >>
>> >> What the uber-hbck (part of hbase-0.90.6 and later releases) does is
>> >> "automate" this meta table surgery to fix the issue.
>> >>
>> >> I will let folks who have tried other repair options respond to you.
>> >> --S
>> >>
>> >> On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
>> >> <[EMAIL PROTECTED]> wrote:
>> >> > Thanks for the response.
>> >> >
>> >> > hbck consistently shows 7 inconsistencies.  I tried -fix (a few
>> times)
>> >> and
>> >> > it didn't seem to fix it.  I tried -details and it does seem to
>> print out
>> >> > more, but nothing super usable from what I can tell.
>> >> >
>> >> > On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]>
>> wrote:
>> >> >
>> >> >> Try doing an hbase hbck to see if it reports inconsistency. And do
>> an
>> >> >> hbase hbck -fix to see if it fixes it for you. See
>> >> >> http://hbase.apache.org/book.html#hbck.in.depth
>> >> >>
>> >> >> Note that since 0.90.4 is old, some of the documented options won't
>> be
>> >> >> available ... but hbase hbck -fix will be available.
>> >> >> --S
>> >> >>
>> >> >> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
>> >> >> <[EMAIL PROTECTED]> wrote:
>> >> >> > Hey all,
>> >> >> >
>> >> >> > I'm running hbase 0.90.4, and somehow the regions for one of my
>> tables
>> >> >> have
>> >> >> > become unlinked from the table itself.  The Master UI shows no
>> regions
>> >> >> for
>> >> >> > the table, and attempting to call Table.getRegionsInfo() returns
>> >> nothing.
>> >> >> >  However, I can see the regions in hdfs in their proper location
>> and
>> >> >> doing
>> >> >> > a scan of the table via the hbase shell does return results.  I
>> also
>> >> get
>> >> >> > results if I do a Get for a specific rowkey.
>> >> >> >
>> >> >> > How can I get these regions back associated with the table, and
>> any
>> >> idea
>> >> >> > why this may have happened?  I tried grepping the hmaster logs but
>> >> didn't
>> >> >> > see anything relevant.
>> >> >> >
>> >> >> > Thanks!
>> >> >> >
>> >> >> > Bryan
>> >> >>
>> >>
>>
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera
>

--
Kevin O'Dell
Customer Operations Engineer, Cloudera

Thanks for the response Kevin, Shrijeet, and Suraj.

I forgot to update the list (sorry!) but the addregion.rb did seem to work.
 At first it didn't seem to, and I stepped away to work on something else
for the last week.  Looking at the table now, it must have been an async
action because all the regions now show up for the table.  I didn't notice
until now because this data was throwaway data that we were going to delete
before noticing the inconsistency.

I didn't save the output from -details, but if this ever happens again I
will try to send it over.  I'll also take a look at your uber-hbck.  Thanks!

On Mon, Jul 9, 2012 at 1:35 PM, Kevin O'dell <[EMAIL PROTECTED]>wrote:

> Bryan,
>
>   I forgot to add the link to the new code to grab the uber-hbck
> http://archive.cloudera.com/cdh/3/hbase-0.90.6-cdh3u4.tar.gz
>
> On Mon, Jul 9, 2012 at 12:53 PM, Kevin O'dell <[EMAIL PROTECTED]
> >wrote:
>
> > Can you please attach the output of hbck -details and the output of
> hadoop
> > fs -lsr /hbase(you can send directly if you do not want to share that)?
> >  There should not be any problem with using the uber-hbck to resolve this
> > from the .90.6 version.
> >
> >
> > On Mon, Jul 2, 2012 at 8:38 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
> >
> >> I was about to propose that next ... you could just try back porting
> >> that uber hbck (or the relevant portions) and run the patched ruby
> >> script ... I've done this for like online meta table merges ... but
> >> not with hbck.
> >>
> >> So - yes, that might be your solve. Good luck!
> >> --S
> >>
> >> On Mon, Jul 2, 2012 at 4:44 PM, Bryan Beaudreault
> >> <[EMAIL PROTECTED]> wrote:
> >> > I'm on cdh3u2 to be precise.  I don't have the fixMeta option, or any
> >> other
> >> > special fix options aside from the basic -fix.
> >> >
> >> > I'll take a look at what 0.90.6 version does when it fixes meta.
>  Maybe
> >> I
> >> > can just follow what it does there.
> >> >
> >> > Thanks again!
> >> >
> >> > On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma <[EMAIL PROTECTED]>
> >> wrote:
> >> >
> >> >> Are you using apache hbase 0.90.4 ... or the one from CDH3?
> >> >> Check what other hbck options you have (do you have -fixMeta?) on the
> >> >> version you are on.
> >> >>
> >> >> What the uber-hbck (part of hbase-0.90.6 and later releases) does is
> >> >> "automate" this meta table surgery to fix the issue.
> >> >>
> >> >> I will let folks who have tried other repair options respond to you.
> >> >> --S
> >> >>
> >> >> On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
> >> >> <[EMAIL PROTECTED]> wrote:
> >> >> > Thanks for the response.
> >> >> >
> >> >> > hbck consistently shows 7 inconsistencies.  I tried -fix (a few
> >> times)
> >> >> and
> >> >> > it didn't seem to fix it.  I tried -details and it does seem to
> >> print out
> >> >> > more, but nothing super usable from what I can tell.
> >> >> >
> >> >> > On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]>
> >> wrote:
> >> >> >
> >> >> >> Try doing an hbase hbck to see if it reports inconsistency. And do
> >> an
> >> >> >> hbase hbck -fix to see if it fixes it for you. See
> >> >> >> http://hbase.apache.org/book.html#hbck.in.depth
> >> >> >>
> >> >> >> Note that since 0.90.4 is old, some of the documented options
> won't
> >> be
> >> >> >> available ... but hbase hbck -fix will be available.
> >> >> >> --S
> >> >> >>
> >> >> >> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
> >> >> >> <[EMAIL PROTECTED]> wrote:
> >> >> >> > Hey all,
> >> >> >> >
> >> >> >> > I'm running hbase 0.90.4, and somehow the regions for one of my
> >> tables
> >> >> >> have
> >> >> >> > become unlinked from the table itself.  The Master UI shows no
> >> regions
> >> >> >> for
> >> >> >> > the table, and attempting to call Table.getRegionsInfo() returns
> >> >> nothing.
> >> >> >> >  However, I can see the regions in hdfs in their proper location
> >> and
> >> >> >> doing
> >> >> >> > a scan of the table via the hbase shell does return results.  I

No problem.  Glad to hear all is well!

On Mon, Jul 9, 2012 at 3:10 PM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> Thanks for the response Kevin, Shrijeet, and Suraj.
>
> I forgot to update the list (sorry!) but the addregion.rb did seem to work.
>  At first it didn't seem to, and I stepped away to work on something else
> for the last week.  Looking at the table now, it must have been an async
> action because all the regions now show up for the table.  I didn't notice
> until now because this data was throwaway data that we were going to delete
> before noticing the inconsistency.
>
> I didn't save the output from -details, but if this ever happens again I
> will try to send it over.  I'll also take a look at your uber-hbck.
>  Thanks!
>
> On Mon, Jul 9, 2012 at 1:35 PM, Kevin O'dell <[EMAIL PROTECTED]
> >wrote:
>
> > Bryan,
> >
> >   I forgot to add the link to the new code to grab the uber-hbck
> > http://archive.cloudera.com/cdh/3/hbase-0.90.6-cdh3u4.tar.gz
> >
> > On Mon, Jul 9, 2012 at 12:53 PM, Kevin O'dell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Can you please attach the output of hbck -details and the output of
> > hadoop
> > > fs -lsr /hbase(you can send directly if you do not want to share that)?
> > >  There should not be any problem with using the uber-hbck to resolve
> this
> > > from the .90.6 version.
> > >
> > >
> > > On Mon, Jul 2, 2012 at 8:38 PM, Suraj Varma <[EMAIL PROTECTED]>
> wrote:
> > >
> > >> I was about to propose that next ... you could just try back porting
> > >> that uber hbck (or the relevant portions) and run the patched ruby
> > >> script ... I've done this for like online meta table merges ... but
> > >> not with hbck.
> > >>
> > >> So - yes, that might be your solve. Good luck!
> > >> --S
> > >>
> > >> On Mon, Jul 2, 2012 at 4:44 PM, Bryan Beaudreault
> > >> <[EMAIL PROTECTED]> wrote:
> > >> > I'm on cdh3u2 to be precise.  I don't have the fixMeta option, or
> any
> > >> other
> > >> > special fix options aside from the basic -fix.
> > >> >
> > >> > I'll take a look at what 0.90.6 version does when it fixes meta.
> >  Maybe
> > >> I
> > >> > can just follow what it does there.
> > >> >
> > >> > Thanks again!
> > >> >
> > >> > On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma <[EMAIL PROTECTED]>
> > >> wrote:
> > >> >
> > >> >> Are you using apache hbase 0.90.4 ... or the one from CDH3?
> > >> >> Check what other hbck options you have (do you have -fixMeta?) on
> the
> > >> >> version you are on.
> > >> >>
> > >> >> What the uber-hbck (part of hbase-0.90.6 and later releases) does
> is
> > >> >> "automate" this meta table surgery to fix the issue.
> > >> >>
> > >> >> I will let folks who have tried other repair options respond to
> you.
> > >> >> --S
> > >> >>
> > >> >> On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
> > >> >> <[EMAIL PROTECTED]> wrote:
> > >> >> > Thanks for the response.
> > >> >> >
> > >> >> > hbck consistently shows 7 inconsistencies.  I tried -fix (a few
> > >> times)
> > >> >> and
> > >> >> > it didn't seem to fix it.  I tried -details and it does seem to
> > >> print out
> > >> >> > more, but nothing super usable from what I can tell.
> > >> >> >
> > >> >> > On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]
> >
> > >> wrote:
> > >> >> >
> > >> >> >> Try doing an hbase hbck to see if it reports inconsistency. And
> do
> > >> an
> > >> >> >> hbase hbck -fix to see if it fixes it for you. See
> > >> >> >> http://hbase.apache.org/book.html#hbck.in.depth
> > >> >> >>
> > >> >> >> Note that since 0.90.4 is old, some of the documented options
> > won't
> > >> be
> > >> >> >> available ... but hbase hbck -fix will be available.
> > >> >> >> --S
> > >> >> >>
> > >> >> >> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
> > >> >> >> <[EMAIL PROTECTED]> wrote:
> > >> >> >> > Hey all,
> > >> >> >> >
> > >> >> >> > I'm running hbase 0.90.4, and somehow the regions for one of
> my
> > >> tables
> > >> >> >> have
> > >> >> >> > become unlinked from the table itself.  The Master UI shows no

Kevin O'Dell
Customer Operations Engineer, Cloudera

Bryan,

Here's the current best documentation about how to use the new features in
hbck.

http://hbase.apache.org/book.html#hbck.in.depth

Jon.

On Mon, Jul 9, 2012 at 12:10 PM, Bryan Beaudreault <[EMAIL PROTECTED]
> wrote:

> Thanks for the response Kevin, Shrijeet, and Suraj.
>
> I forgot to update the list (sorry!) but the addregion.rb did seem to work.
>  At first it didn't seem to, and I stepped away to work on something else
> for the last week.  Looking at the table now, it must have been an async
> action because all the regions now show up for the table.  I didn't notice
> until now because this data was throwaway data that we were going to delete
> before noticing the inconsistency.
>
> I didn't save the output from -details, but if this ever happens again I
> will try to send it over.  I'll also take a look at your uber-hbck.
>  Thanks!
>
> On Mon, Jul 9, 2012 at 1:35 PM, Kevin O'dell <[EMAIL PROTECTED]
> >wrote:
>
> > Bryan,
> >
> >   I forgot to add the link to the new code to grab the uber-hbck
> > http://archive.cloudera.com/cdh/3/hbase-0.90.6-cdh3u4.tar.gz
> >
> > On Mon, Jul 9, 2012 at 12:53 PM, Kevin O'dell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Can you please attach the output of hbck -details and the output of
> > hadoop
> > > fs -lsr /hbase(you can send directly if you do not want to share that)?
> > >  There should not be any problem with using the uber-hbck to resolve
> this
> > > from the .90.6 version.
> > >
> > >
> > > On Mon, Jul 2, 2012 at 8:38 PM, Suraj Varma <[EMAIL PROTECTED]>
> wrote:
> > >
> > >> I was about to propose that next ... you could just try back porting
> > >> that uber hbck (or the relevant portions) and run the patched ruby
> > >> script ... I've done this for like online meta table merges ... but
> > >> not with hbck.
> > >>
> > >> So - yes, that might be your solve. Good luck!
> > >> --S
> > >>
> > >> On Mon, Jul 2, 2012 at 4:44 PM, Bryan Beaudreault
> > >> <[EMAIL PROTECTED]> wrote:
> > >> > I'm on cdh3u2 to be precise.  I don't have the fixMeta option, or
> any
> > >> other
> > >> > special fix options aside from the basic -fix.
> > >> >
> > >> > I'll take a look at what 0.90.6 version does when it fixes meta.
> >  Maybe
> > >> I
> > >> > can just follow what it does there.
> > >> >
> > >> > Thanks again!
> > >> >
> > >> > On Mon, Jul 2, 2012 at 7:36 PM, Suraj Varma <[EMAIL PROTECTED]>
> > >> wrote:
> > >> >
> > >> >> Are you using apache hbase 0.90.4 ... or the one from CDH3?
> > >> >> Check what other hbck options you have (do you have -fixMeta?) on
> the
> > >> >> version you are on.
> > >> >>
> > >> >> What the uber-hbck (part of hbase-0.90.6 and later releases) does
> is
> > >> >> "automate" this meta table surgery to fix the issue.
> > >> >>
> > >> >> I will let folks who have tried other repair options respond to
> you.
> > >> >> --S
> > >> >>
> > >> >> On Mon, Jul 2, 2012 at 4:24 PM, Bryan Beaudreault
> > >> >> <[EMAIL PROTECTED]> wrote:
> > >> >> > Thanks for the response.
> > >> >> >
> > >> >> > hbck consistently shows 7 inconsistencies.  I tried -fix (a few
> > >> times)
> > >> >> and
> > >> >> > it didn't seem to fix it.  I tried -details and it does seem to
> > >> print out
> > >> >> > more, but nothing super usable from what I can tell.
> > >> >> >
> > >> >> > On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]
> >
> > >> wrote:
> > >> >> >
> > >> >> >> Try doing an hbase hbck to see if it reports inconsistency. And
> do
> > >> an
> > >> >> >> hbase hbck -fix to see if it fixes it for you. See
> > >> >> >> http://hbase.apache.org/book.html#hbck.in.depth
> > >> >> >>
> > >> >> >> Note that since 0.90.4 is old, some of the documented options
> > won't
> > >> be
> > >> >> >> available ... but hbase hbck -fix will be available.
> > >> >> >> --S
> > >> >> >>
> > >> >> >> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
> > >> >> >> <[EMAIL PROTECTED]> wrote:
> > >> >> >> > Hey all,
> > >> >> >> >
> > >> >> >
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]

Actually I just noticed part that I must have skimmed over.  It  mentions
that I have 3 regions for this table not in .META. but deployed on servers.
 Interestingly there are 4 other regions for other tables that are on HDFS
but not in META or deployed on any server.

Can I manually add these somehow?  Also this only mentions 3 of the regions
for the table in my initial question.  There are still over 300 other
regions listed in hdfs but not showing in the ui and such.

On Mon, Jul 2, 2012 at 7:24 PM, Bryan Beaudreault
<[EMAIL PROTECTED]>wrote:

> Thanks for the response.
>
> hbck consistently shows 7 inconsistencies.  I tried -fix (a few times) and
> it didn't seem to fix it.  I tried -details and it does seem to print out
> more, but nothing super usable from what I can tell.
>
> On Mon, Jul 2, 2012 at 6:32 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>
>> Try doing an hbase hbck to see if it reports inconsistency. And do an
>> hbase hbck -fix to see if it fixes it for you. See
>> http://hbase.apache.org/book.html#hbck.in.depth
>>
>> Note that since 0.90.4 is old, some of the documented options won't be
>> available ... but hbase hbck -fix will be available.
>> --S
>>
>> On Mon, Jul 2, 2012 at 7:34 AM, Bryan Beaudreault
>> <[EMAIL PROTECTED]> wrote:
>> > Hey all,
>> >
>> > I'm running hbase 0.90.4, and somehow the regions for one of my tables
>> have
>> > become unlinked from the table itself.  The Master UI shows no regions
>> for
>> > the table, and attempting to call Table.getRegionsInfo() returns
>> nothing.
>> >  However, I can see the regions in hdfs in their proper location and
>> doing
>> > a scan of the table via the hbase shell does return results.  I also get
>> > results if I do a Get for a specific rowkey.
>> >
>> > How can I get these regions back associated with the table, and any idea
>> > why this may have happened?  I tried grepping the hmaster logs but
>> didn't
>> > see anything relevant.
>> >
>> > Thanks!
>> >
>> > Bryan
>>
>
>