|
|
-
Prewarming in-memory column families
Eric Czech 2013-02-22, 15:39
Hi everyone,
Are blocks for in-memory column families automatically loaded in to the block cache on restart? If not, would anyone recommend running a scan with .setCacheBlocks(true) after a restart for in-memory column families?
Thanks!
-
Re: Prewarming in-memory column families
Stack 2013-02-26, 22:13
On Fri, Feb 22, 2013 at 7:39 AM, Eric Czech <[EMAIL PROTECTED]> wrote:
> Hi everyone, > > Are blocks for in-memory column families automatically loaded in to the > block cache on restart? No > If not, would anyone recommend running a scan with > .setCacheBlocks(true) after a restart for in-memory column families? >
Yes.
It should be easy verifying whether the above warmup had an effect.
Good luck, St.Ack
-
Re: Prewarming in-memory column families
Sergey Shelukhin 2013-02-27, 01:04
should we make this built-in? Sounds like default user intent for in-memory.
On Tue, Feb 26, 2013 at 2:13 PM, Stack <[EMAIL PROTECTED]> wrote:
> On Fri, Feb 22, 2013 at 7:39 AM, Eric Czech <[EMAIL PROTECTED]> wrote: > > > Hi everyone, > > > > Are blocks for in-memory column families automatically loaded in to the > > block cache on restart? > > > No > > > > If not, would anyone recommend running a scan with > > .setCacheBlocks(true) after a restart for in-memory column families? > > > > Yes. > > It should be easy verifying whether the above warmup had an effect. > > Good luck, > St.Ack >
-
Re: Prewarming in-memory column families
Jean-Daniel Cryans 2013-02-27, 01:16
It's touchy... what if the data set doesn't fit in the in-memory's part of the block cache (which is 25%)? Maybe the user only wants to keep "in-memory" those edits that are being used? What about the IO hit of assigning those regions at startup that would now need to read X GBs all at once?
FWIW I've never been a fan of that setting.
J-D
On Tue, Feb 26, 2013 at 5:04 PM, Sergey Shelukhin <[EMAIL PROTECTED]> wrote: > should we make this built-in? Sounds like default user intent for in-memory. > > On Tue, Feb 26, 2013 at 2:13 PM, Stack <[EMAIL PROTECTED]> wrote: > >> On Fri, Feb 22, 2013 at 7:39 AM, Eric Czech <[EMAIL PROTECTED]> wrote: >> >> > Hi everyone, >> > >> > Are blocks for in-memory column families automatically loaded in to the >> > block cache on restart? >> >> >> No >> >> >> > If not, would anyone recommend running a scan with >> > .setCacheBlocks(true) after a restart for in-memory column families? >> > >> >> Yes. >> >> It should be easy verifying whether the above warmup had an effect. >> >> Good luck, >> St.Ack >>
-
Re: Prewarming in-memory column families
Eric Czech 2013-03-01, 16:07
Thanks guys.
I think I'll use the initial scan then but limit the rows used to a smaller subset that I know should fit in the in-memory part of the block cache.
On Tue, Feb 26, 2013 at 8:16 PM, Jean-Daniel Cryans <[EMAIL PROTECTED]>wrote:
> It's touchy... what if the data set doesn't fit in the in-memory's > part of the block cache (which is 25%)? Maybe the user only wants to > keep "in-memory" those edits that are being used? What about the IO > hit of assigning those regions at startup that would now need to read > X GBs all at once? > > FWIW I've never been a fan of that setting. > > J-D > > On Tue, Feb 26, 2013 at 5:04 PM, Sergey Shelukhin > <[EMAIL PROTECTED]> wrote: > > should we make this built-in? Sounds like default user intent for > in-memory. > > > > On Tue, Feb 26, 2013 at 2:13 PM, Stack <[EMAIL PROTECTED]> wrote: > > > >> On Fri, Feb 22, 2013 at 7:39 AM, Eric Czech <[EMAIL PROTECTED]> wrote: > >> > >> > Hi everyone, > >> > > >> > Are blocks for in-memory column families automatically loaded in to > the > >> > block cache on restart? > >> > >> > >> No > >> > >> > >> > If not, would anyone recommend running a scan with > >> > .setCacheBlocks(true) after a restart for in-memory column families? > >> > > >> > >> Yes. > >> > >> It should be easy verifying whether the above warmup had an effect. > >> > >> Good luck, > >> St.Ack > >> >
|
|