Hello list,

what are your experience with having more than one HBase-instance per
machine?

One reason to do this could be that you got i.e. 32 GB RAM and want to
give 12-14 GB of RAM to each HBase instance so that you got little
issues with Garbage Collection while using the available RAM-capacity
for caching.

One thing I worry about is replication of data. Say two of three
replicas reside at the same box in two distinct HBase instances.
If this box fails for some reason, the risk of loosing data is
relatively high compared to a scenario where all three replicas are
stored at different nodes.

Kind regards,
Em

On Fri, Jun 15, 2012 at 10:50 AM, Em <[EMAIL PROTECTED]> wrote:
> what are your experience with having more than one HBase-instance per
> machine?
>

My experience is ittle.

Its been done though by others on this list.

> One reason to do this could be that you got i.e. 32 GB RAM and want to
> give 12-14 GB of RAM to each HBase instance so that you got little
> issues with Garbage Collection while using the available RAM-capacity
> for caching.
>

You should leave some RAM for the file system cache.

Are you unable to tame your GCs?  Is that why the two servers per node?

> One thing I worry about is replication of data. Say two of three
> replicas reside at the same box in two distinct HBase instances.

So, you intend doubling the datanode instances per machine too?

St.Ack

Stack,

I have no issues with HBase, the question is purely theoretical.

> So, you intend doubling the datanode instances per machine too?
Everything else would not make sense to me, or what do you think?

Thanks for your feedback!

Regards,
Em

Am 16.06.2012 07:12, schrieb Stack:
> On Fri, Jun 15, 2012 at 10:50 AM, Em <[EMAIL PROTECTED]> wrote:
>> what are your experience with having more than one HBase-instance per
>> machine?
>>
>
> My experience is ittle.
>
> Its been done though by others on this list.
>
>> One reason to do this could be that you got i.e. 32 GB RAM and want to
>> give 12-14 GB of RAM to each HBase instance so that you got little
>> issues with Garbage Collection while using the available RAM-capacity
>> for caching.
>>
>
> You should leave some RAM for the file system cache.
>
> Are you unable to tame your GCs?  Is that why the two servers per node?
>
>> One thing I worry about is replication of data. Say two of three
>> replicas reside at the same box in two distinct HBase instances.
>
> So, you intend doubling the datanode instances per machine too?
>
> St.Ack
>

Hi,

I have done this at a customer site to overcome the 0.90.x slow WAL performance. With one RS per DN we bottlenecked, with 5-7 RS per DN we were able to hit the target rate.

Please note that we did this in lieu of the proper built-in options like WAL compression, multiple WAL, or n-way writes for WALs.

Lars

On Jun 16, 2012, at 10:53, Em <[EMAIL PROTECTED]> wrote:

> Stack,
>
> I have no issues with HBase, the question is purely theoretical.
>
>> So, you intend doubling the datanode instances per machine too?
> Everything else would not make sense to me, or what do you think?
>
> Thanks for your feedback!
>
> Regards,
> Em
>
> Am 16.06.2012 07:12, schrieb Stack:
>> On Fri, Jun 15, 2012 at 10:50 AM, Em <[EMAIL PROTECTED]> wrote:
>>> what are your experience with having more than one HBase-instance per
>>> machine?
>>>
>>
>> My experience is ittle.
>>
>> Its been done though by others on this list.
>>
>>> One reason to do this could be that you got i.e. 32 GB RAM and want to
>>> give 12-14 GB of RAM to each HBase instance so that you got little
>>> issues with Garbage Collection while using the available RAM-capacity
>>> for caching.
>>>
>>
>> You should leave some RAM for the file system cache.
>>
>> Are you unable to tame your GCs?  Is that why the two servers per node?
>>
>>> One thing I worry about is replication of data. Say two of three
>>> replicas reside at the same box in two distinct HBase instances.
>>
>> So, you intend doubling the datanode instances per machine too?
>>
>> St.Ack
>>