|
|
-
Duplicates region after upgrade to 0.89
Andrey Stepachev 2010-09-27, 13:42
After upgrade from 0.20.6 to 0.89 (stumbleupon version)
I mentioned, that i have duplicate entries in .META.
(encoded names are different, but keys are identical)
check_meta.rb sad :
What I can do to recover from this situation? Export/Import only?
And if that, I must export/import production database to be able to
use 0.89?
.... skip ....
REGION => {NAME =>
'bsn.main,\x00\x00\x00\x00\x00G\x1A\x9A,1285148769688.4fbcb0d5a09063998ebf3894552a91f0.',
STARTKEY => '\x00\x00\x00\x00\x00G\x1A\x9A', ENDKEY =>
'\x00\x00\x00\x00\x00J\x9Cd',
ENCODED => 4fbcb0d5a09063998ebf3894552a91f0,
REGION => {NAME =>
'bsn.main,\x00\x00\x00\x00\x00G\x1A\x9A,1285157746310.aaeb272c3d8563f2bb6cebb487ddc10e.',
STARTKEY => '\x00\x00\x00\x00\x00G\x1A\x9A', ENDKEY =>
'\x00\x00\x00\x00\x00J\x9Cd',
ENCODED => aaeb272c3d8563f2bb6cebb487ddc10e,
REGION => {NAME =>
'bsn.main,\x00\x00\x00\x00\x00J\x9Cd,1285148769688.49fa4f03cccfd17f44dcae99d443676c.',
STARTKEY => '\x00\x00\x00\x00\x00J\x9Cd', ENDKEY =>
'\x00\x00\x00\x00\x00N\x14\xBA',
ENCODED => 49fa4f03cccfd17f44dcae99d443676c,
REGION => {NAME =>
'bsn.main,\x00\x00\x00\x00\x00J\x9Cd,1285157746310.11d235841dc91ae49c6c474027c34f31.',
STARTKEY => '\x00\x00\x00\x00\x00J\x9Cd', ENDKEY =>
'\x00\x00\x00\x00\x00N\x14\xBA',
ENCODED => 11d235841dc91ae49c6c474027c34f31,
REGION => {NAME =>
'bsn.main,\x00\x00\x00\x00\x00N\x14\xBA,1283127663502.1023102751',
STARTKEY => '\x00\x00\x00\x00\x00N\x14\xBA', ENDKEY =>
'\x00\x00\x00\x00\x00V\xDDZ',
ENCODED => 1023102751,
.... skip ....
check_mata logs:
check_meta: hole after REGION => {NAME =>
'bsn.main,\x00\x00\x00\x00\x00G\x1A\x9A,1285148769688.4fbcb0d5a09063998ebf3894552a91f0.',
STARTKEY => '\x00\x00\x00\x00\x00G\x1A\x9A', ENDKEY =>
'\x00\x00\x00\x00\x00J\x9Cd', ENCODED =>
4fbcb0d5a09063998ebf3894552a91f0 }
issuing with --fix flag doesn't help.
Andrey.
-
Re: Duplicates region after upgrade to 0.89
Stack 2010-09-27, 16:42
Can you scan your table? (Try a count from the shell or use rowcounter
mapreduce job).
What happens when you do check_meta.rb --fix?
Can you do a:
$ echo "scan '.META.'" | ./bin/hbase shell &> /tmp/meta.txt
... and pastebin the /tmp/meta.txt so I can take a look?
St.Ack
On Mon, Sep 27, 2010 at 6:42 AM, Andrey Stepachev <[EMAIL PROTECTED]> wrote:
> After upgrade from 0.20.6 to 0.89 (stumbleupon version)
> I mentioned, that i have duplicate entries in .META.
> (encoded names are different, but keys are identical)
>
> check_meta.rb sad :
>
>
> What I can do to recover from this situation? Export/Import only?
> And if that, I must export/import production database to be able to
> use 0.89?
>
> .... skip ....
> REGION => {NAME =>
> 'bsn.main,\x00\x00\x00\x00\x00G\x1A\x9A,1285148769688.4fbcb0d5a09063998ebf3894552a91f0.',
> STARTKEY => '\x00\x00\x00\x00\x00G\x1A\x9A', ENDKEY =>
> '\x00\x00\x00\x00\x00J\x9Cd',
> ENCODED => 4fbcb0d5a09063998ebf3894552a91f0,
> REGION => {NAME =>
> 'bsn.main,\x00\x00\x00\x00\x00G\x1A\x9A,1285157746310.aaeb272c3d8563f2bb6cebb487ddc10e.',
> STARTKEY => '\x00\x00\x00\x00\x00G\x1A\x9A', ENDKEY =>
> '\x00\x00\x00\x00\x00J\x9Cd',
> ENCODED => aaeb272c3d8563f2bb6cebb487ddc10e,
> REGION => {NAME =>
> 'bsn.main,\x00\x00\x00\x00\x00J\x9Cd,1285148769688.49fa4f03cccfd17f44dcae99d443676c.',
> STARTKEY => '\x00\x00\x00\x00\x00J\x9Cd', ENDKEY =>
> '\x00\x00\x00\x00\x00N\x14\xBA',
> ENCODED => 49fa4f03cccfd17f44dcae99d443676c,
> REGION => {NAME =>
> 'bsn.main,\x00\x00\x00\x00\x00J\x9Cd,1285157746310.11d235841dc91ae49c6c474027c34f31.',
> STARTKEY => '\x00\x00\x00\x00\x00J\x9Cd', ENDKEY =>
> '\x00\x00\x00\x00\x00N\x14\xBA',
> ENCODED => 11d235841dc91ae49c6c474027c34f31,
> REGION => {NAME =>
> 'bsn.main,\x00\x00\x00\x00\x00N\x14\xBA,1283127663502.1023102751',
> STARTKEY => '\x00\x00\x00\x00\x00N\x14\xBA', ENDKEY =>
> '\x00\x00\x00\x00\x00V\xDDZ',
> ENCODED => 1023102751,
> .... skip ....
>
> check_mata logs:
> check_meta: hole after REGION => {NAME =>
> 'bsn.main,\x00\x00\x00\x00\x00G\x1A\x9A,1285148769688.4fbcb0d5a09063998ebf3894552a91f0.',
> STARTKEY => '\x00\x00\x00\x00\x00G\x1A\x9A', ENDKEY =>
> '\x00\x00\x00\x00\x00J\x9Cd', ENCODED =>
> 4fbcb0d5a09063998ebf3894552a91f0 }
>
> issuing with --fix flag doesn't help.
>
> Andrey.
>
-
Re: Duplicates region after upgrade to 0.89
Andrey Stepachev 2010-09-27, 18:40
Thanks for reply Stack. How data was imported and chaged (full story). 1. stop 0.20.6. 2. copy /hbase into new location (where 0.89 lives) 3. start 0.89 4. run heavy update task .... scan output: http://paste.ubuntu.com/501680/--fix output: http://paste.ubuntu.com/501684/Row count is bigger then should. ~6.3mil instead of 5.4mil (looks like this regions are copy of data was writer on update, but instead of write to existing region, hbase create two new regions) 2010/9/27 Stack <[EMAIL PROTECTED]>: > Can you scan your table? (Try a count from the shell or use rowcounter > mapreduce job). > > What happens when you do check_meta.rb --fix? > > Can you do a: > > $ echo "scan '.META.'" | ./bin/hbase shell &> /tmp/meta.txt > > ... and pastebin the /tmp/meta.txt so I can take a look? > > St.Ack > > On Mon, Sep 27, 2010 at 6:42 AM, Andrey Stepachev <[EMAIL PROTECTED]> wrote: >> After upgrade from 0.20.6 to 0.89 (stumbleupon version) >> I mentioned, that i have duplicate entries in .META. >> (encoded names are different, but keys are identical) >> >> check_meta.rb sad : >> >> >> What I can do to recover from this situation? Export/Import only? >> And if that, I must export/import production database to be able to >> use 0.89? >> >> >> Andrey. >> >
-
Re: Duplicates region after upgrade to 0.89
Andrey Stepachev 2010-09-27, 19:24
this is restored data (before any upates and all region looks good) http://paste.ubuntu.com/501716/2010/9/27 Andrey Stepachev <[EMAIL PROTECTED]>: > Thanks for reply Stack. > > How data was imported and chaged (full story). > 1. stop 0.20.6. > 2. copy /hbase into new location (where 0.89 lives) > 3. start 0.89 > 4. run heavy update task > .... > > scan output: > http://paste.ubuntu.com/501680/> > --fix output: > http://paste.ubuntu.com/501684/> > Row count is bigger then should. ~6.3mil instead of 5.4mil > (looks like this regions are copy of data was writer on update, > but instead of write to existing region, hbase create two new regions) > > 2010/9/27 Stack <[EMAIL PROTECTED]>: >> Can you scan your table? (Try a count from the shell or use rowcounter >> mapreduce job). >> >> What happens when you do check_meta.rb --fix? >> >> Can you do a: >> >> $ echo "scan '.META.'" | ./bin/hbase shell &> /tmp/meta.txt >> >> ... and pastebin the /tmp/meta.txt so I can take a look? >> >> St.Ack >> >> On Mon, Sep 27, 2010 at 6:42 AM, Andrey Stepachev <[EMAIL PROTECTED]> wrote: >>> After upgrade from 0.20.6 to 0.89 (stumbleupon version) >>> I mentioned, that i have duplicate entries in .META. >>> (encoded names are different, but keys are identical) >>> >>> check_meta.rb sad : >>> >>> >>> What I can do to recover from this situation? Export/Import only? >>> And if that, I must export/import production database to be able to >>> use 0.89? >>> >>> >>> Andrey. >>> >> >
|
|
