HBase, mail # user - holes in tables.

holes in tables.

Jack Levin 2011-12-25, 23:54

Greetings all. How does one deals with holes in tables between
regions nowadays?
Name Region Server Start Key End Key
img644,,1317474152909.02f379ab6f08f4d7609ef1245cb7033a. not deployed 1cce.jpg
img644,1cce.jpg,1317474152909.ebb8778fc1e67965c518e357125678ea. not
deployed 1cce.jpg 1udf.jpg
img644,3s3b.jpg,1321061240287.42bbb43e5c06d7b708ee51de97c7b126. not
deployed 3s3b.jpg 69yl.jpg
img644,69yl.jpg,1321061243702.f49886c58ec17d9158b9effc1693c496. not
deployed 69yl.jpg 8yo.jpg

Note, I got "1cce.jpg 1udf.jpg" region followed by "3s3b.jpg 69yl.jpg";

Doing

1. disable 'img644'
2.. running add_table.rb on 'img644'

Re: holes in tables.

Jack Levin 2011-12-25, 23:55

3. enable 'img644'
Does not solve the problem.

-Jack

On Sun, Dec 25, 2011 at 3:54 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
> Greetings all. How does one deals with holes in tables between
> regions nowadays?
>
>
> Name Region Server Start Key End Key
> img644,,1317474152909.02f379ab6f08f4d7609ef1245cb7033a. not deployed 1cce.jpg
> img644,1cce.jpg,1317474152909.ebb8778fc1e67965c518e357125678ea. not
> deployed 1cce.jpg 1udf.jpg
> img644,3s3b.jpg,1321061240287.42bbb43e5c06d7b708ee51de97c7b126. not
> deployed 3s3b.jpg 69yl.jpg
> img644,69yl.jpg,1321061243702.f49886c58ec17d9158b9effc1693c496. not
> deployed 69yl.jpg 8yo.jpg
>
> Note, I got "1cce.jpg 1udf.jpg" region followed by "3s3b.jpg 69yl.jpg";
>
> Doing
>
> 1. disable 'img644'
> 2.. running add_table.rb on 'img644'

Re: holes in tables.

Ted Yu 2011-12-26, 00:39

Which version of HBase ?
I guess 0.90.4 ?

Cheers

On Sun, Dec 25, 2011 at 3:55 PM, Jack Levin <[EMAIL PROTECTED]> wrote:

> 3. enable 'img644'
>
>
> Does not solve the problem.
>
> -Jack
>
> On Sun, Dec 25, 2011 at 3:54 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
> > Greetings all. How does one deals with holes in tables between
> > regions nowadays?
> >
> >
> > Name Region Server Start Key End Key
> > img644,,1317474152909.02f379ab6f08f4d7609ef1245cb7033a. not deployed
> 1cce.jpg
> > img644,1cce.jpg,1317474152909.ebb8778fc1e67965c518e357125678ea. not
> > deployed 1cce.jpg 1udf.jpg
> > img644,3s3b.jpg,1321061240287.42bbb43e5c06d7b708ee51de97c7b126. not
> > deployed 3s3b.jpg 69yl.jpg
> > img644,69yl.jpg,1321061243702.f49886c58ec17d9158b9effc1693c496. not
> > deployed 69yl.jpg 8yo.jpg
> >
> > Note, I got "1cce.jpg 1udf.jpg" region followed by "3s3b.jpg
> 69yl.jpg";
> >
> > Doing
> >
> > 1. disable 'img644'
> > 2.. running add_table.rb on 'img644'
>

Re: holes in tables.

Jack Levin 2011-12-26, 03:15

Yep

-Jack
On Dec 25, 2011, at 4:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Which version of HBase ?
> I guess 0.90.4 ?
>
> Cheers
>
> On Sun, Dec 25, 2011 at 3:55 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>
>> 3. enable 'img644'
>>
>>
>> Does not solve the problem.
>>
>> -Jack
>>
>> On Sun, Dec 25, 2011 at 3:54 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>> Greetings all. How does one deals with holes in tables between
>>> regions nowadays?
>>>
>>>
>>> Name Region Server Start Key End Key
>>> img644,,1317474152909.02f379ab6f08f4d7609ef1245cb7033a. not deployed
>> 1cce.jpg
>>> img644,1cce.jpg,1317474152909.ebb8778fc1e67965c518e357125678ea. not
>>> deployed 1cce.jpg 1udf.jpg
>>> img644,3s3b.jpg,1321061240287.42bbb43e5c06d7b708ee51de97c7b126. not
>>> deployed 3s3b.jpg 69yl.jpg
>>> img644,69yl.jpg,1321061243702.f49886c58ec17d9158b9effc1693c496. not
>>> deployed 69yl.jpg 8yo.jpg
>>>
>>> Note, I got "1cce.jpg 1udf.jpg" region followed by "3s3b.jpg
>> 69yl.jpg";
>>>
>>> Doing
>>>
>>> 1. disable 'img644'
>>> 2.. running add_table.rb on 'img644'
>>

Re: holes in tables.

Ted Yu 2011-12-26, 03:23

OfflineMetaRepair class is in 0.90.5
See HBASE-4377

But maybe you're looking for an online solution.

On Sun, Dec 25, 2011 at 7:15 PM, Jack Levin <[EMAIL PROTECTED]> wrote:

> Yep
>
> -Jack
>
>
> On Dec 25, 2011, at 4:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Which version of HBase ?
> > I guess 0.90.4 ?
> >
> > Cheers
> >
> > On Sun, Dec 25, 2011 at 3:55 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
> >
> >> 3. enable 'img644'
> >>
> >>
> >> Does not solve the problem.
> >>
> >> -Jack
> >>
> >> On Sun, Dec 25, 2011 at 3:54 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
> >>> Greetings all. How does one deals with holes in tables between
> >>> regions nowadays?
> >>>
> >>>
> >>> Name Region Server Start Key End Key
> >>> img644,,1317474152909.02f379ab6f08f4d7609ef1245cb7033a. not deployed
> >> 1cce.jpg
> >>> img644,1cce.jpg,1317474152909.ebb8778fc1e67965c518e357125678ea. not
> >>> deployed 1cce.jpg 1udf.jpg
> >>> img644,3s3b.jpg,1321061240287.42bbb43e5c06d7b708ee51de97c7b126. not
> >>> deployed 3s3b.jpg 69yl.jpg
> >>> img644,69yl.jpg,1321061243702.f49886c58ec17d9158b9effc1693c496. not
> >>> deployed 69yl.jpg 8yo.jpg
> >>>
> >>> Note, I got "1cce.jpg 1udf.jpg" region followed by "3s3b.jpg
> >> 69yl.jpg";
> >>>
> >>> Doing
> >>>
> >>> 1. disable 'img644'
> >>> 2.. running add_table.rb on 'img644'
> >>
>

Re: holes in tables.

Lars George 2011-12-26, 09:49

You could also have a go at https://issues.apache.org/jira/browse/HBASE-4009

Simple script based on the check_meta.rb to patch holes.

Lars

On Dec 26, 2011, at 4:23 AM, Ted Yu wrote:

> OfflineMetaRepair class is in 0.90.5
> See HBASE-4377
>
> But maybe you're looking for an online solution.
>
> On Sun, Dec 25, 2011 at 7:15 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>
>> Yep
>>
>> -Jack
>>
>>
>> On Dec 25, 2011, at 4:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>>> Which version of HBase ?
>>> I guess 0.90.4 ?
>>>
>>> Cheers
>>>
>>> On Sun, Dec 25, 2011 at 3:55 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>>
>>>> 3. enable 'img644'
>>>>
>>>>
>>>> Does not solve the problem.
>>>>
>>>> -Jack
>>>>
>>>> On Sun, Dec 25, 2011 at 3:54 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>>>> Greetings all. How does one deals with holes in tables between
>>>>> regions nowadays?
>>>>>
>>>>>
>>>>> Name Region Server Start Key End Key
>>>>> img644,,1317474152909.02f379ab6f08f4d7609ef1245cb7033a. not deployed
>>>> 1cce.jpg
>>>>> img644,1cce.jpg,1317474152909.ebb8778fc1e67965c518e357125678ea. not
>>>>> deployed 1cce.jpg 1udf.jpg
>>>>> img644,3s3b.jpg,1321061240287.42bbb43e5c06d7b708ee51de97c7b126. not
>>>>> deployed 3s3b.jpg 69yl.jpg
>>>>> img644,69yl.jpg,1321061243702.f49886c58ec17d9158b9effc1693c496. not
>>>>> deployed 69yl.jpg 8yo.jpg
>>>>>
>>>>> Note, I got "1cce.jpg 1udf.jpg" region followed by "3s3b.jpg
>>>> 69yl.jpg";
>>>>>
>>>>> Doing
>>>>>
>>>>> 1. disable 'img644'
>>>>> 2.. running add_table.rb on 'img644'
>>>>
>>

Re: holes in tables.

Jack Levin 2011-12-26, 18:15

patch_meta.rb has some problems. I ran it, and it would create region
start/end combos, there were incorrect in terms of order, for example
"dab - cab", which is a region that is not right, because first key is
greater than end key. I guess what I really need is a very simple
tool, that takes something like this:

# add_empty_region.rb <table> <first key> <end key>

This way I can fill in the gap/whole, and be on my merry way. Right
now I resort to merging regions with hadoop fs -cp, which is a pain in
a butt if there are too many of them to merge.

Best,

-Jack

On Mon, Dec 26, 2011 at 1:49 AM, Lars George <[EMAIL PROTECTED]> wrote:
> You could also have a go at https://issues.apache.org/jira/browse/HBASE-4009
>
> Simple script based on the check_meta.rb to patch holes.
>
> Lars
>
> On Dec 26, 2011, at 4:23 AM, Ted Yu wrote:
>
>> OfflineMetaRepair class is in 0.90.5
>> See HBASE-4377
>>
>> But maybe you're looking for an online solution.
>>
>> On Sun, Dec 25, 2011 at 7:15 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>
>>> Yep
>>>
>>> -Jack
>>>
>>>
>>> On Dec 25, 2011, at 4:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>>
>>>> Which version of HBase ?
>>>> I guess 0.90.4 ?
>>>>
>>>> Cheers
>>>>
>>>> On Sun, Dec 25, 2011 at 3:55 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> 3. enable 'img644'
>>>>>
>>>>>
>>>>> Does not solve the problem.
>>>>>
>>>>> -Jack
>>>>>
>>>>> On Sun, Dec 25, 2011 at 3:54 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>>>>> Greetings all. How does one deals with holes in tables between
>>>>>> regions nowadays?
>>>>>>
>>>>>>
>>>>>> Name Region Server Start Key End Key
>>>>>> img644,,1317474152909.02f379ab6f08f4d7609ef1245cb7033a. not deployed
>>>>> 1cce.jpg
>>>>>> img644,1cce.jpg,1317474152909.ebb8778fc1e67965c518e357125678ea. not
>>>>>> deployed 1cce.jpg 1udf.jpg
>>>>>> img644,3s3b.jpg,1321061240287.42bbb43e5c06d7b708ee51de97c7b126. not
>>>>>> deployed 3s3b.jpg 69yl.jpg
>>>>>> img644,69yl.jpg,1321061243702.f49886c58ec17d9158b9effc1693c496. not
>>>>>> deployed 69yl.jpg 8yo.jpg
>>>>>>
>>>>>> Note, I got "1cce.jpg 1udf.jpg" region followed by "3s3b.jpg
>>>>> 69yl.jpg";
>>>>>>
>>>>>> Doing
>>>>>>
>>>>>> 1. disable 'img644'
>>>>>> 2.. running add_table.rb on 'img644'
>>>>>
>>>
>

Re: holes in tables.

Lars George 2011-12-26, 18:28

Hi Jack,

Have a look into the JRuby source of that script. It is trivial to patch it so that it does what you want.

Lars

On Dec 26, 2011, at 19:15, Jack Levin <[EMAIL PROTECTED]> wrote:

> patch_meta.rb has some problems. I ran it, and it would create region
> start/end combos, there were incorrect in terms of order, for example
> "dab - cab", which is a region that is not right, because first key is
> greater than end key. I guess what I really need is a very simple
> tool, that takes something like this:
>
> # add_empty_region.rb <table> <first key> <end key>
>
> This way I can fill in the gap/whole, and be on my merry way. Right
> now I resort to merging regions with hadoop fs -cp, which is a pain in
> a butt if there are too many of them to merge.
>
> Best,
>
> -Jack
>
> On Mon, Dec 26, 2011 at 1:49 AM, Lars George <[EMAIL PROTECTED]> wrote:
>> You could also have a go at https://issues.apache.org/jira/browse/HBASE-4009
>>
>> Simple script based on the check_meta.rb to patch holes.
>>
>> Lars
>>
>> On Dec 26, 2011, at 4:23 AM, Ted Yu wrote:
>>
>>> OfflineMetaRepair class is in 0.90.5
>>> See HBASE-4377
>>>
>>> But maybe you're looking for an online solution.
>>>
>>> On Sun, Dec 25, 2011 at 7:15 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>>
>>>> Yep
>>>>
>>>> -Jack
>>>>
>>>>
>>>> On Dec 25, 2011, at 4:39 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Which version of HBase ?
>>>>> I guess 0.90.4 ?
>>>>>
>>>>> Cheers
>>>>>
>>>>> On Sun, Dec 25, 2011 at 3:55 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> 3. enable 'img644'
>>>>>>
>>>>>>
>>>>>> Does not solve the problem.
>>>>>>
>>>>>> -Jack
>>>>>>
>>>>>> On Sun, Dec 25, 2011 at 3:54 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>>>>>> Greetings all. How does one deals with holes in tables between
>>>>>>> regions nowadays?
>>>>>>>
>>>>>>>
>>>>>>> Name Region Server Start Key End Key
>>>>>>> img644,,1317474152909.02f379ab6f08f4d7609ef1245cb7033a. not deployed
>>>>>> 1cce.jpg
>>>>>>> img644,1cce.jpg,1317474152909.ebb8778fc1e67965c518e357125678ea. not
>>>>>>> deployed 1cce.jpg 1udf.jpg
>>>>>>> img644,3s3b.jpg,1321061240287.42bbb43e5c06d7b708ee51de97c7b126. not
>>>>>>> deployed 3s3b.jpg 69yl.jpg
>>>>>>> img644,69yl.jpg,1321061243702.f49886c58ec17d9158b9effc1693c496. not
>>>>>>> deployed 69yl.jpg 8yo.jpg
>>>>>>>
>>>>>>> Note, I got "1cce.jpg 1udf.jpg" region followed by "3s3b.jpg
>>>>>> 69yl.jpg";
>>>>>>>
>>>>>>> Doing
>>>>>>>
>>>>>>> 1. disable 'img644'
>>>>>>> 2.. running add_table.rb on 'img644'
>>>>>>
>>>>
>>