|
|
-
Re: Checking major compaction
kiran 2012-10-18, 06:33
Thanks ram,
Is there a way can I check it through region server logs. If it is possible
what are the statements that I need to look for ??
Thanks
Kiran
On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
[EMAIL PROTECTED]> wrote:
> HBASE-6033 does the work that you ask for. It is currently in Trunk
> version
> of HBase.
>
> Regards
> Ram
>
> > -----Original Message-----
> > From: kiran [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, October 18, 2012 11:43 AM
> > To: [EMAIL PROTECTED]
> > Subject: Checking major compaction
> >
> > Hi all,
> >
> > Is there a way to check if major compaction is running or not on a
> > table.
> >
> > --
> > Thank you
> > Kiran Sarvabhotla
> >
> > -----Even a correct decision is wrong when it is taken late
>
>
--
Thank you
Kiran Sarvabhotla
-----Even a correct decision is wrong when it is taken late
+
kiran 2012-10-18, 06:33
-
RE: Checking major compaction
Ramkrishna.S.Vasudevan 2012-10-18, 08:34
Hi
Yes Kiran you can go thro the logs also.
You will see some logs like
'Start major compaction for ..............
'Compacting file ................
'Compacting file ................
And finally 'Completed major/minor compaction.....'
I just don have some exact logs with me right now. But you can see log msgs
but all comes in debug mode. So ensure you enable debug mode for your logs.
A simple test would be to just right some 10 rows. In between do some 4 to 5
flushes.
Just give major_compact(tableName) from the shell. You can see the logs.
:)
Regards
Ram
> -----Original Message-----
> From: kiran [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, October 18, 2012 12:03 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Checking major compaction
>
> Thanks ram,
>
> Is there a way can I check it through region server logs. If it is
> possible
> what are the statements that I need to look for ??
>
> Thanks
> Kiran
>
> On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> [EMAIL PROTECTED]> wrote:
>
> > HBASE-6033 does the work that you ask for. It is currently in Trunk
> > version
> > of HBase.
> >
> > Regards
> > Ram
> >
> > > -----Original Message-----
> > > From: kiran [mailto:[EMAIL PROTECTED]]
> > > Sent: Thursday, October 18, 2012 11:43 AM
> > > To: [EMAIL PROTECTED]
> > > Subject: Checking major compaction
> > >
> > > Hi all,
> > >
> > > Is there a way to check if major compaction is running or not on a
> > > table.
> > >
> > > --
> > > Thank you
> > > Kiran Sarvabhotla
> > >
> > > -----Even a correct decision is wrong when it is taken late
> >
> >
>
>
> --
> Thank you
> Kiran Sarvabhotla
>
> -----Even a correct decision is wrong when it is taken late
+
Ramkrishna.S.Vasudevan 2012-10-18, 08:34
-
RE: Checking major compaction
Ramkrishna.S.Vasudevan 2012-10-18, 09:25
>A simple test would be to just right some 10 rows
I meant to say write some 10 rows.(not right)
Regards
Ram
> -----Original Message-----
> From: Ramkrishna.S.Vasudevan [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, October 18, 2012 2:05 PM
> To: [EMAIL PROTECTED]
> Subject: RE: Checking major compaction
>
> Hi
>
> Yes Kiran you can go thro the logs also.
>
> You will see some logs like
> 'Start major compaction for ..............
> 'Compacting file ................
> 'Compacting file ................
> And finally 'Completed major/minor compaction.....'
>
> I just don have some exact logs with me right now. But you can see log
> msgs
> but all comes in debug mode. So ensure you enable debug mode for your
> logs.
>
> A simple test would be to just right some 10 rows. In between do some 4
> to 5
> flushes.
>
> Just give major_compact(tableName) from the shell. You can see the
> logs.
> :)
>
> Regards
> Ram
>
> > -----Original Message-----
> > From: kiran [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, October 18, 2012 12:03 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Checking major compaction
> >
> > Thanks ram,
> >
> > Is there a way can I check it through region server logs. If it is
> > possible
> > what are the statements that I need to look for ??
> >
> > Thanks
> > Kiran
> >
> > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > HBASE-6033 does the work that you ask for. It is currently in
> Trunk
> > > version
> > > of HBase.
> > >
> > > Regards
> > > Ram
> > >
> > > > -----Original Message-----
> > > > From: kiran [mailto:[EMAIL PROTECTED]]
> > > > Sent: Thursday, October 18, 2012 11:43 AM
> > > > To: [EMAIL PROTECTED]
> > > > Subject: Checking major compaction
> > > >
> > > > Hi all,
> > > >
> > > > Is there a way to check if major compaction is running or not on
> a
> > > > table.
> > > >
> > > > --
> > > > Thank you
> > > > Kiran Sarvabhotla
> > > >
> > > > -----Even a correct decision is wrong when it is taken late
> > >
> > >
> >
> >
> > --
> > Thank you
> > Kiran Sarvabhotla
> >
> > -----Even a correct decision is wrong when it is taken late
+
Ramkrishna.S.Vasudevan 2012-10-18, 09:25
-
Checking major compaction
kiran 2012-10-18, 06:12
Hi all,
Is there a way to check if major compaction is running or not on a table.
--
Thank you
Kiran Sarvabhotla
-----Even a correct decision is wrong when it is taken late
+
kiran 2012-10-18, 06:12
-
RE: Checking major compaction
Ramkrishna.S.Vasudevan 2012-10-18, 06:25
HBASE-6033 does the work that you ask for. It is currently in Trunk version
of HBase.
Regards
Ram
> -----Original Message-----
> From: kiran [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, October 18, 2012 11:43 AM
> To: [EMAIL PROTECTED]
> Subject: Checking major compaction
>
> Hi all,
>
> Is there a way to check if major compaction is running or not on a
> table.
>
> --
> Thank you
> Kiran Sarvabhotla
>
> -----Even a correct decision is wrong when it is taken late
+
Ramkrishna.S.Vasudevan 2012-10-18, 06:25
-
Re: Checking major compaction
kiran 2012-10-19, 15:37
Hi all,
I mistakenly execution major compaction on the entire table instead of
region by region. It is causing so much delay in the write performance,
also on the disk and networkI/O. How can i stop major compaction now ?
Thanks
Kiran
On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan <
[EMAIL PROTECTED]> wrote:
> >A simple test would be to just right some 10 rows
> I meant to say write some 10 rows.(not right)
>
> Regards
> Ram
>
> > -----Original Message-----
> > From: Ramkrishna.S.Vasudevan [mailto:[EMAIL PROTECTED]]
> > Sent: Thursday, October 18, 2012 2:05 PM
> > To: [EMAIL PROTECTED]
> > Subject: RE: Checking major compaction
> >
> > Hi
> >
> > Yes Kiran you can go thro the logs also.
> >
> > You will see some logs like
> > 'Start major compaction for ..............
> > 'Compacting file ................
> > 'Compacting file ................
> > And finally 'Completed major/minor compaction.....'
> >
> > I just don have some exact logs with me right now. But you can see log
> > msgs
> > but all comes in debug mode. So ensure you enable debug mode for your
> > logs.
> >
> > A simple test would be to just right some 10 rows. In between do some 4
> > to 5
> > flushes.
> >
> > Just give major_compact(tableName) from the shell. You can see the
> > logs.
> > :)
> >
> > Regards
> > Ram
> >
> > > -----Original Message-----
> > > From: kiran [mailto:[EMAIL PROTECTED]]
> > > Sent: Thursday, October 18, 2012 12:03 PM
> > > To: [EMAIL PROTECTED]
> > > Subject: Re: Checking major compaction
> > >
> > > Thanks ram,
> > >
> > > Is there a way can I check it through region server logs. If it is
> > > possible
> > > what are the statements that I need to look for ??
> > >
> > > Thanks
> > > Kiran
> > >
> > > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > HBASE-6033 does the work that you ask for. It is currently in
> > Trunk
> > > > version
> > > > of HBase.
> > > >
> > > > Regards
> > > > Ram
> > > >
> > > > > -----Original Message-----
> > > > > From: kiran [mailto:[EMAIL PROTECTED]]
> > > > > Sent: Thursday, October 18, 2012 11:43 AM
> > > > > To: [EMAIL PROTECTED]
> > > > > Subject: Checking major compaction
> > > > >
> > > > > Hi all,
> > > > >
> > > > > Is there a way to check if major compaction is running or not on
> > a
> > > > > table.
> > > > >
> > > > > --
> > > > > Thank you
> > > > > Kiran Sarvabhotla
> > > > >
> > > > > -----Even a correct decision is wrong when it is taken late
> > > >
> > > >
> > >
> > >
> > > --
> > > Thank you
> > > Kiran Sarvabhotla
> > >
> > > -----Even a correct decision is wrong when it is taken late
>
>
--
Thank you
Kiran Sarvabhotla
-----Even a correct decision is wrong when it is taken late
+
kiran 2012-10-19, 15:37
-
Re: Checking major compaction
Kevin O'dell 2012-10-19, 15:46
Kiran,
I don't believe you can. If you have a sane region count/region size it
should be over soon.
On Fri, Oct 19, 2012 at 11:37 AM, kiran <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I mistakenly execution major compaction on the entire table instead of
> region by region. It is causing so much delay in the write performance,
> also on the disk and networkI/O. How can i stop major compaction now ?
>
> Thanks
> Kiran
>
> On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan <
> [EMAIL PROTECTED]> wrote:
>
> > >A simple test would be to just right some 10 rows
> > I meant to say write some 10 rows.(not right)
> >
> > Regards
> > Ram
> >
> > > -----Original Message-----
> > > From: Ramkrishna.S.Vasudevan [mailto:[EMAIL PROTECTED]]
> > > Sent: Thursday, October 18, 2012 2:05 PM
> > > To: [EMAIL PROTECTED]
> > > Subject: RE: Checking major compaction
> > >
> > > Hi
> > >
> > > Yes Kiran you can go thro the logs also.
> > >
> > > You will see some logs like
> > > 'Start major compaction for ..............
> > > 'Compacting file ................
> > > 'Compacting file ................
> > > And finally 'Completed major/minor compaction.....'
> > >
> > > I just don have some exact logs with me right now. But you can see log
> > > msgs
> > > but all comes in debug mode. So ensure you enable debug mode for your
> > > logs.
> > >
> > > A simple test would be to just right some 10 rows. In between do some 4
> > > to 5
> > > flushes.
> > >
> > > Just give major_compact(tableName) from the shell. You can see the
> > > logs.
> > > :)
> > >
> > > Regards
> > > Ram
> > >
> > > > -----Original Message-----
> > > > From: kiran [mailto:[EMAIL PROTECTED]]
> > > > Sent: Thursday, October 18, 2012 12:03 PM
> > > > To: [EMAIL PROTECTED]
> > > > Subject: Re: Checking major compaction
> > > >
> > > > Thanks ram,
> > > >
> > > > Is there a way can I check it through region server logs. If it is
> > > > possible
> > > > what are the statements that I need to look for ??
> > > >
> > > > Thanks
> > > > Kiran
> > > >
> > > > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > HBASE-6033 does the work that you ask for. It is currently in
> > > Trunk
> > > > > version
> > > > > of HBase.
> > > > >
> > > > > Regards
> > > > > Ram
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: kiran [mailto:[EMAIL PROTECTED]]
> > > > > > Sent: Thursday, October 18, 2012 11:43 AM
> > > > > > To: [EMAIL PROTECTED]
> > > > > > Subject: Checking major compaction
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Is there a way to check if major compaction is running or not on
> > > a
> > > > > > table.
> > > > > >
> > > > > > --
> > > > > > Thank you
> > > > > > Kiran Sarvabhotla
> > > > > >
> > > > > > -----Even a correct decision is wrong when it is taken late
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Thank you
> > > > Kiran Sarvabhotla
> > > >
> > > > -----Even a correct decision is wrong when it is taken late
> >
> >
>
>
> --
> Thank you
> Kiran Sarvabhotla
>
> -----Even a correct decision is wrong when it is taken late
>
--
Kevin O'Dell
Customer Operations Engineer, Cloudera
+
Kevin O'dell 2012-10-19, 15:46
-
Re: Checking major compaction
kiran 2012-10-19, 16:10
Kevin,
Thanks for your reply,
can restarting region servers help to kill the already existing compaction
processes in region servers..
Thanks
Kiran
On Fri, Oct 19, 2012 at 9:16 PM, Kevin O'dell <[EMAIL PROTECTED]>wrote:
> Kiran,
>
> I don't believe you can. If you have a sane region count/region size it
> should be over soon.
>
> On Fri, Oct 19, 2012 at 11:37 AM, kiran <[EMAIL PROTECTED]>
> wrote:
>
> > Hi all,
> >
> > I mistakenly execution major compaction on the entire table instead of
> > region by region. It is causing so much delay in the write performance,
> > also on the disk and networkI/O. How can i stop major compaction now ?
> >
> > Thanks
> > Kiran
> >
> > On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > >A simple test would be to just right some 10 rows
> > > I meant to say write some 10 rows.(not right)
> > >
> > > Regards
> > > Ram
> > >
> > > > -----Original Message-----
> > > > From: Ramkrishna.S.Vasudevan [mailto:[EMAIL PROTECTED]
> ]
> > > > Sent: Thursday, October 18, 2012 2:05 PM
> > > > To: [EMAIL PROTECTED]
> > > > Subject: RE: Checking major compaction
> > > >
> > > > Hi
> > > >
> > > > Yes Kiran you can go thro the logs also.
> > > >
> > > > You will see some logs like
> > > > 'Start major compaction for ..............
> > > > 'Compacting file ................
> > > > 'Compacting file ................
> > > > And finally 'Completed major/minor compaction.....'
> > > >
> > > > I just don have some exact logs with me right now. But you can see
> log
> > > > msgs
> > > > but all comes in debug mode. So ensure you enable debug mode for
> your
> > > > logs.
> > > >
> > > > A simple test would be to just right some 10 rows. In between do
> some 4
> > > > to 5
> > > > flushes.
> > > >
> > > > Just give major_compact(tableName) from the shell. You can see the
> > > > logs.
> > > > :)
> > > >
> > > > Regards
> > > > Ram
> > > >
> > > > > -----Original Message-----
> > > > > From: kiran [mailto:[EMAIL PROTECTED]]
> > > > > Sent: Thursday, October 18, 2012 12:03 PM
> > > > > To: [EMAIL PROTECTED]
> > > > > Subject: Re: Checking major compaction
> > > > >
> > > > > Thanks ram,
> > > > >
> > > > > Is there a way can I check it through region server logs. If it is
> > > > > possible
> > > > > what are the statements that I need to look for ??
> > > > >
> > > > > Thanks
> > > > > Kiran
> > > > >
> > > > > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
> > > > > [EMAIL PROTECTED]> wrote:
> > > > >
> > > > > > HBASE-6033 does the work that you ask for. It is currently in
> > > > Trunk
> > > > > > version
> > > > > > of HBase.
> > > > > >
> > > > > > Regards
> > > > > > Ram
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: kiran [mailto:[EMAIL PROTECTED]]
> > > > > > > Sent: Thursday, October 18, 2012 11:43 AM
> > > > > > > To: [EMAIL PROTECTED]
> > > > > > > Subject: Checking major compaction
> > > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > Is there a way to check if major compaction is running or not
> on
> > > > a
> > > > > > > table.
> > > > > > >
> > > > > > > --
> > > > > > > Thank you
> > > > > > > Kiran Sarvabhotla
> > > > > > >
> > > > > > > -----Even a correct decision is wrong when it is taken late
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thank you
> > > > > Kiran Sarvabhotla
> > > > >
> > > > > -----Even a correct decision is wrong when it is taken late
> > >
> > >
> >
> >
> > --
> > Thank you
> > Kiran Sarvabhotla
> >
> > -----Even a correct decision is wrong when it is taken late
> >
>
>
>
> --
> Kevin O'Dell
> Customer Operations Engineer, Cloudera
>
--
Thank you
Kiran Sarvabhotla
-----Even a correct decision is wrong when it is taken late
+
kiran 2012-10-19, 16:10
-
Re: Checking major compaction
Jean-Daniel Cryans 2012-10-19, 16:11
You could just disable then enable the table.
J-D
On Fri, Oct 19, 2012 at 9:10 AM, kiran <[EMAIL PROTECTED]> wrote:
> Kevin,
>
> Thanks for your reply,
>
> can restarting region servers help to kill the already existing compaction
> processes in region servers..
>
> Thanks
> Kiran
>
> On Fri, Oct 19, 2012 at 9:16 PM, Kevin O'dell <[EMAIL PROTECTED]>wrote:
>
>> Kiran,
>>
>> I don't believe you can. If you have a sane region count/region size it
>> should be over soon.
>>
>> On Fri, Oct 19, 2012 at 11:37 AM, kiran <[EMAIL PROTECTED]>
>> wrote:
>>
>> > Hi all,
>> >
>> > I mistakenly execution major compaction on the entire table instead of
>> > region by region. It is causing so much delay in the write performance,
>> > also on the disk and networkI/O. How can i stop major compaction now ?
>> >
>> > Thanks
>> > Kiran
>> >
>> > On Thu, Oct 18, 2012 at 2:55 PM, Ramkrishna.S.Vasudevan <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> > > >A simple test would be to just right some 10 rows
>> > > I meant to say write some 10 rows.(not right)
>> > >
>> > > Regards
>> > > Ram
>> > >
>> > > > -----Original Message-----
>> > > > From: Ramkrishna.S.Vasudevan [mailto:[EMAIL PROTECTED]
>> ]
>> > > > Sent: Thursday, October 18, 2012 2:05 PM
>> > > > To: [EMAIL PROTECTED]
>> > > > Subject: RE: Checking major compaction
>> > > >
>> > > > Hi
>> > > >
>> > > > Yes Kiran you can go thro the logs also.
>> > > >
>> > > > You will see some logs like
>> > > > 'Start major compaction for ..............
>> > > > 'Compacting file ................
>> > > > 'Compacting file ................
>> > > > And finally 'Completed major/minor compaction.....'
>> > > >
>> > > > I just don have some exact logs with me right now. But you can see
>> log
>> > > > msgs
>> > > > but all comes in debug mode. So ensure you enable debug mode for
>> your
>> > > > logs.
>> > > >
>> > > > A simple test would be to just right some 10 rows. In between do
>> some 4
>> > > > to 5
>> > > > flushes.
>> > > >
>> > > > Just give major_compact(tableName) from the shell. You can see the
>> > > > logs.
>> > > > :)
>> > > >
>> > > > Regards
>> > > > Ram
>> > > >
>> > > > > -----Original Message-----
>> > > > > From: kiran [mailto:[EMAIL PROTECTED]]
>> > > > > Sent: Thursday, October 18, 2012 12:03 PM
>> > > > > To: [EMAIL PROTECTED]
>> > > > > Subject: Re: Checking major compaction
>> > > > >
>> > > > > Thanks ram,
>> > > > >
>> > > > > Is there a way can I check it through region server logs. If it is
>> > > > > possible
>> > > > > what are the statements that I need to look for ??
>> > > > >
>> > > > > Thanks
>> > > > > Kiran
>> > > > >
>> > > > > On Thu, Oct 18, 2012 at 11:55 AM, Ramkrishna.S.Vasudevan <
>> > > > > [EMAIL PROTECTED]> wrote:
>> > > > >
>> > > > > > HBASE-6033 does the work that you ask for. It is currently in
>> > > > Trunk
>> > > > > > version
>> > > > > > of HBase.
>> > > > > >
>> > > > > > Regards
>> > > > > > Ram
>> > > > > >
>> > > > > > > -----Original Message-----
>> > > > > > > From: kiran [mailto:[EMAIL PROTECTED]]
>> > > > > > > Sent: Thursday, October 18, 2012 11:43 AM
>> > > > > > > To: [EMAIL PROTECTED]
>> > > > > > > Subject: Checking major compaction
>> > > > > > >
>> > > > > > > Hi all,
>> > > > > > >
>> > > > > > > Is there a way to check if major compaction is running or not
>> on
>> > > > a
>> > > > > > > table.
>> > > > > > >
>> > > > > > > --
>> > > > > > > Thank you
>> > > > > > > Kiran Sarvabhotla
>> > > > > > >
>> > > > > > > -----Even a correct decision is wrong when it is taken late
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Thank you
>> > > > > Kiran Sarvabhotla
>> > > > >
>> > > > > -----Even a correct decision is wrong when it is taken late
>> > >
>> > >
>> >
>> >
>> > --
>> > Thank you
>> > Kiran Sarvabhotla
>> >
>> > -----Even a correct decision is wrong when it is taken late
>> >
>>
>>
>>
>> --
>> Kevin O'Dell
+
Jean-Daniel Cryans 2012-10-19, 16:11
|
|
