Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Never ending major compaction?


+
Jean-Marc Spaggiari 2013-01-01, 19:42
+
Ted 2013-01-01, 21:08
+
Jean-Marc Spaggiari 2013-01-02, 02:14
+
Ted Yu 2013-01-02, 06:07
+
Adrien Mogenet 2013-01-02, 07:49
Copy link to this message
-
Re: Never ending major compaction?
When I'm doing the compaction it's because one region moved to a new
server, and that's what was never ending.

I was not able to reproduce that. I will continue to keep an eye on
this and will provide logs and details if it happens again.

2013/1/2, Adrien Mogenet <[EMAIL PROTECTED]>:
> Do you have 100% data locality ? How is the network trafic ?
>
>
> On Wed, Jan 2, 2013 at 7:07 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> bq. It took about 6h to complete.
>>
>> If the above behavior is reproducible, we should investigate more deeply.
>>
>> Thanks for sharing.
>>
>> On Tue, Jan 1, 2013 at 6:14 PM, Jean-Marc Spaggiari <
>> [EMAIL PROTECTED]
>> > wrote:
>>
>> > Yes, I'm running on 0.94.3. The last major compaction ran yesterday.
>> > It's almost daily. That's why I was surprise it took so long. I mean,
>> > I'm only compacting regions who moved. So it should be pretty quick.
>> > But was not the case. It took about 6h to complete. Strange. Maybe
>> > something went wrong when I stopped/started hbase.
>> >
>> > also, there was almost no activity on the network nor on the CPUs. I
>> > will have to add disks monitoring on Ganglia to see if I was limited
>> > by IOs...
>> >
>> > I looked at the regions logs and everything was fine. It was showing
>> > some compaction information every few seconds.
>> >
>> > JM
>> >
>> > 2013/1/1, Ted <[EMAIL PROTECTED]>:
>> > > You're on hbase 0.94.3 , right ?
>> > >
>> > > When was the last time major compaction ran ?
>> > >
>> > > Compaction is region server activity so you should be able to find
>> > > some
>> > clue
>> > > in region server log.
>> > >
>> > > Cheers
>> > >
>> > > On Jan 1, 2013, at 11:42 AM, Jean-Marc Spaggiari <
>> > [EMAIL PROTECTED]>
>> > > wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> I have a table, with about 200 000 raws, keys is string[64] (ish)
>> > >> and
>> > >> value is string[512].
>> > >>
>> > >> It's splitted over 16 regions located on 7 regionservers.
>> > >>
>> > >> So it's not a big table, and there is a lot of horsepower behind it.
>> > >>
>> > >> I asked a major_compaction few hours ago. Let's say, about 5 hours
>> > >> ago. and it's still compacting! But all servers activities seems to
>> > >> be
>> > >> null. CPU usage is almost 0.
>> > >>
>> > >> There is nothing on the master logs.
>> > >>
>> > >> How can I see what's going on? Is there a way to see the compaction
>> > >> queue?
>> > >>
>> > >> JM
>> > >
>> >
>>
>
>
>
> --
> Adrien Mogenet
> 06.59.16.64.22
> http://www.mogenet.me
>