Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Issue with hbase


Copy link to this message
-
Re: Issue with hbase
Irwan,

  Just to be clear: You don't have the META, ROOT, or any .regioninfo
files, but the storefiles are still intact?

If so, you should be able to recreate your tables using the correct HBase
home and bulk load your data back in?  I think that would work, but I will
defer to Stack or Michael on this one.

On Wed, Jul 25, 2012 at 2:28 AM, Irwan Hadi <[EMAIL PROTECTED]> wrote:

> A few weeks ago we installed hbase 0.90.5 on our dev system, to play
> around with it, along with Hadoop, and HDFS. All of these are
> installed on a Centos 6 server.
> Even though we specified hbase to use HDFS in hbase-site.xml, we had a
> small configuration typo which then apparently caused it to default
> storing data to /tmp/hbase-hadoop/hbase
>
> As expected, after running for a couple days, the tmpwatch cron
> cleaned up certain files in /tmp.  The issue wasn't apparent until
> hbase had to be cycled, which shows that we have 0 tables.
>
> The question now is:
> We still have the tables intact, but lost the META, ROOT, and
> .regioninfo under /tmp/hbase-hadoop/hbase/<table_name>/id/<number> ,
> are these enough to somehow recover the tables, if so how can we do
> this?
> We tried hbase hbck already, but it does not want to fix the issue
>
> ======================================================================================> 12/07/25 06:23:11 INFO zookeeper.ClientCnxn: EventThread shut down
> Number of Tables: 0
> Number of live region servers: 1
> Number of dead region servers: 0
> Number of empty REGIONINFO_QUALIFIER rows in .META.: 0
> 12/07/25 06:23:11 INFO zookeeper.ZooKeeper: Initiating client
> connection, connectString=localhost:2181 sessionTimeout=180000
> watcher=hconnection
> 12/07/25 06:23:11 INFO zookeeper.ClientCnxn: Opening socket connection
> to server localhost/127.0.0.1:2181
> 12/07/25 06:23:11 INFO zookeeper.ClientCnxn: Socket connection
> established to localhost/127.0.0.1:2181, initiating session
> 12/07/25 06:23:11 INFO zookeeper.ClientCnxn: Session establishment
> complete on server localhost/127.0.0.1:2181, sessionid > 0x138bca7fff5000a, negotiated timeout = 40000
> 12/07/25 06:23:11 INFO
> client.HConnectionManager$HConnectionImplementation: Closed zookeeper
> sessionid=0x138bca7fff5000a
> 12/07/25 06:23:11 INFO zookeeper.ZooKeeper: Session: 0x138bca7fff5000a
> closed
> 12/07/25 06:23:11 INFO zookeeper.ClientCnxn: EventThread shut down
> 12/07/25 06:23:11 DEBUG
> client.HConnectionManager$HConnectionImplementation: The connection to
> null has been closed.
> ERROR: Region
> file:/tmp/hbase-hadoop/hbase/news/349a8254e3fe134eb0ec82693e8ff905
> on HDFS, but not listed in META or deployed on any region server
> ERROR: Region
> file:/tmp/hbase-hadoop/hbase/story/89770459431a8e7558d952fc4ee7c60b
> on HDFS, but not listed in META or deployed on any region server
> ERROR: Region
> file:/tmp/hbase-hadoop/hbase/topic/cb411aee8e80c70b03b0ebdc574ba8ad
> on HDFS, but not listed in META or deployed on any region server
> Summary:
>   -ROOT- is okay.
>     Number of regions: 1
>     Deployed on:  web01.dev.:52768
>   .META. is okay.
>     Number of regions: 1
>     Deployed on:  web01.dev.:52768
> 3 inconsistencies detected.
> Status: INCONSISTENT
>
> ======================================================================================>
>
>
> We also tried rebuilding the META using the following "hbase
> org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair -base
> file:///tmp/hbase-hadoop/hbase/" , but got the following error:
>
> ======================================================================================> 12/07/25 05:25:58 INFO util.HBaseFsck: Loading HBase regioninfo from
> HDFS...
> 12/07/25 05:25:59 DEBUG util.HBaseFsck: HRegionInfo read: REGION =>
> {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED =>
> 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', FAMILIES
> => [{NAME => 'info', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
> VERSIONS => '10', COMPRESSION => 'NONE', TTL => '2147483647',

Kevin O'Dell
Customer Operations Engineer, Cloudera