How about a slightly larger value, say 10MB ?

Cheers

On Mon, Dec 5, 2011 at 9:47 AM, Stack <[EMAIL PROTECTED]> wrote:

> On Sun, Dec 4, 2011 at 10:08 PM, Lars <[EMAIL PROTECTED]> wrote:
> > Lastly, XX:MaxDirectMemorySize should definitely be documented.
> >
>
> I can doc this.  Should we set it to 1MB(?) in our default jvm configs
> (seems like default is 64M according to
>
> http://stackoverflow.com/questions/3773775/default-for-xxmaxdirectmemorysize
> )?
>
> Nice work lads,
> St.Ack
>

On Mon, Dec 5, 2011 at 9:51 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> How about a slightly larger value, say 10MB ?
>

Sure.  Let me put a patch up and include it in next RC.
St.Ack

The off-heap cache will also depend on a high value for this setting..
need to make sure there isn't some kind of conflict between this
feature and whatever we choose as a default.

-Todd

On Mon, Dec 5, 2011 at 9:53 AM, Stack <[EMAIL PROTECTED]> wrote:
> On Mon, Dec 5, 2011 at 9:51 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> How about a slightly larger value, say 10MB ?
>>
>
> Sure.  Let me put a patch up and include it in next RC.
> St.Ack

--
Todd Lipcon
Software Engineer, Cloudera

Thanks for the reminder Todd.
Currently we use the following config param to calculate offheap cache size:
    long offHeapCacheSize       (long) (conf.getFloat("hbase.offheapcache.percentage", (float) 0.95) *
          DirectMemoryUtils.getDirectMemorySize());
We should lower the default percentage for offheap cache and document the
tradeoff between giving larger space from direct memory to either offheap
cache or client NIO.

Cheers

On Mon, Dec 5, 2011 at 10:04 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> The off-heap cache will also depend on a high value for this setting..
> need to make sure there isn't some kind of conflict between this
> feature and whatever we choose as a default.
>
> -Todd
>
> On Mon, Dec 5, 2011 at 9:53 AM, Stack <[EMAIL PROTECTED]> wrote:
> > On Mon, Dec 5, 2011 at 9:51 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >> How about a slightly larger value, say 10MB ?
> >>
> >
> > Sure.  Let me put a patch up and include it in next RC.
> > St.Ack
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

If the problem is only the client, then why don't we suggest different
settings for client vs server?

I imagine most clients aren't even using the HBASE_OPTS parameter,
right? Only the shell... in which case these settings in hbase-env
won't have any effect.

We should probably default the offheapcache to 0% and make people
explicitly enable it, IMO.

-Todd

On Mon, Dec 5, 2011 at 10:22 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> Thanks for the reminder Todd.
> Currently we use the following config param to calculate offheap cache size:
>    long offHeapCacheSize >      (long) (conf.getFloat("hbase.offheapcache.percentage", (float) 0.95) *
>          DirectMemoryUtils.getDirectMemorySize());
> We should lower the default percentage for offheap cache and document the
> tradeoff between giving larger space from direct memory to either offheap
> cache or client NIO.
>
> Cheers
>
> On Mon, Dec 5, 2011 at 10:04 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>
>> The off-heap cache will also depend on a high value for this setting..
>> need to make sure there isn't some kind of conflict between this
>> feature and whatever we choose as a default.
>>
>> -Todd
>>
>> On Mon, Dec 5, 2011 at 9:53 AM, Stack <[EMAIL PROTECTED]> wrote:
>> > On Mon, Dec 5, 2011 at 9:51 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >> How about a slightly larger value, say 10MB ?
>> >>
>> >
>> > Sure.  Let me put a patch up and include it in next RC.
>> > St.Ack
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>

--
Todd Lipcon
Software Engineer, Cloudera

On Mon, Dec 5, 2011 at 10:43 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> If the problem is only the client, then why don't we suggest different
> settings for client vs server?
>
> I imagine most clients aren't even using the HBASE_OPTS parameter,
> right? Only the shell... in which case these settings in hbase-env
> won't have any effect.
>
> We should probably default the offheapcache to 0% and make people
> explicitly enable it, IMO.
>

Let me work up an FAQ for client running out of memory suggesting set
an upper bound on clients (that can be different from server where
offheap caching might be going on).

St.Ack

If you don't mind, please add to Troubleshooting (as opposed to FAQ)

There's a Client section in the Troubleshooting chapter already.

http://hbase.apache.org/book.html#trouble.client
On 12/5/11 3:12 PM, "Stack" <[EMAIL PROTECTED]> wrote:

>On Mon, Dec 5, 2011 at 10:43 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>> If the problem is only the client, then why don't we suggest different
>> settings for client vs server?
>>
>> I imagine most clients aren't even using the HBASE_OPTS parameter,
>> right? Only the shell... in which case these settings in hbase-env
>> won't have any effect.
>>
>> We should probably default the offheapcache to 0% and make people
>> explicitly enable it, IMO.
>>
>
>Let me work up an FAQ for client running out of memory suggesting set
>an upper bound on clients (that can be different from server where
>offheap caching might be going on).
>
>St.Ack
>