Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Merging regions issue


Copy link to this message
-
Re: Merging regions issue
Ok. Seems that I'm facing HBASE-1212....

The only issue is that when the merge is failing, hbck become inconsistent.

JM

2012/12/5, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
> Hi everyone,
>
> Sorry, I did not figured I was only replying to Marcos ;)
>
> So here are more details about this issue.
>
> I'm using HBase 0.94.3 and Hadoop 1.0.3.
>
> The merge seems to be failing when there is to many merges done.
>
> I just gave it another try... Each time I'm doing a major_compact
> before trying the merges. And hbck.
>
> Build the table with 4 regions and 1000 rows. Keys are 8 bytes long
> and value is 512 bytes. Everything went well, and hbck is not
> reporting additionnal errors.
>
> Another try with 16 regions and 10000 rows worked well too.
>
> Another try with 54 regions and 10000 rows but this one is not working
> fine. I'm getting some errors and the hbck is giving that:
> ERROR: Orphan region in HDFS: Unable to load .regioninfo from table
> testtable in hdfs dir
> hdfs://node3:9000/hbase/testtable/88203ca27c9beedb02004d93e7181f94!
> It may be an invalid format or version file.  Treating as an orphaned
> regiondir.
> ERROR: Orphan region in HDFS: Unable to load .regioninfo from table
> testtable in hdfs dir
> hdfs://node3:9000/hbase/testtable/c359655af1e7beb8138123e8aed4c382!
> It may be an invalid format or version file.  Treating as an orphaned
> regiondir.
> ERROR: Orphan region in HDFS: Unable to load .regioninfo from table
> testtable in hdfs dir
> hdfs://node3:9000/hbase/testtable/e537a847f8c5a549993001b2bb9c0102!
> It may be an invalid format or version file.  Treating as an orphaned
> regiondir.
> ERROR: Orphan region in HDFS: Unable to load .regioninfo from table
> testtable in hdfs dir
> hdfs://node3:9000/hbase/testtable/ef558e802a90b493677b5c07325b12fd!
> It may be an invalid format or version file.  Treating as an orphaned
> regiondir.
>
> ERROR: Region { meta => null, hdfs =>
> hdfs://node3:9000/hbase/testtable/88203ca27c9beedb02004d93e7181f94,
> deployed =>  } on HDFS, but not listed in META or deployed on any
> region server
> ERROR: Region { meta => null, hdfs =>
> hdfs://node3:9000/hbase/testtable/c359655af1e7beb8138123e8aed4c382,
> deployed =>  } on HDFS, but not listed in META or deployed on any
> region server
> ERROR: Region { meta => null, hdfs =>
> hdfs://node3:9000/hbase/testtable/e537a847f8c5a549993001b2bb9c0102,
> deployed =>  } on HDFS, but not listed in META or deployed on any
> region server
> ERROR: Region { meta => null, hdfs =>
> hdfs://node3:9000/hbase/testtable/ef558e802a90b493677b5c07325b12fd,
> deployed =>  } on HDFS, but not listed in META or deployed on any
> region server
>
> And below is the log from the merge application. After that it's
> listing all the regions in the server (a lot).
>
> I'm not sure if it's the first occurance of the issue or not.
>
> Should I open a JIRA for that? It's difficult to reproduce because
> it's not a fixed pattern, but I still can get is failing easily.
>
> I will activate the DEBUG logs on the HRegion class, and give it again
> and again another try ;)
>
> Thanks,
>
> JM
>
> Merging
> testtable,?\xEC\x1B\x90^\xDB\xC9\xA5,1354727393540.3dca6fdbf95546ac71f47403a047fd10.
> with
> testtable,?\xEC\xA0_\xCE+\xB7),1354727402292.a5a42e51f3414fc45d77c1759378b58e.
> 12/12/05 12:18:56 INFO util.Merge: Verifying that file system is
> available...
> 12/12/05 12:18:56 INFO util.Merge: Verifying that HBase is not running...
> 12/12/05 12:18:56 INFO zookeeper.ZooKeeper: Initiating client
> connection, connectString=latitude:2181,cube:2181,node3:2181
> sessionTimeout=180000 watcher=hconnection
> 12/12/05 12:18:56 INFO zookeeper.ClientCnxn: Opening socket connection
> to server /192.168.23.1:2181
> 12/12/05 12:18:56 INFO zookeeper.RecoverableZooKeeper: The identifier
> of this process is 13131@node3
> 12/12/05 12:18:56 INFO client.ZooKeeperSaslClient: Client will not
> SASL-authenticate because the default JAAS configuration section
> 'Client' could not be found. If you are not using SASL, you may ignore