Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Multiple HBase instances per machine


+
Em 2012-06-15, 17:50
+
Stack 2012-06-16, 05:12
+
Em 2012-06-16, 08:53
Copy link to this message
-
Re: Multiple HBase instances per machine
Lars George 2012-06-16, 13:51
Hi,

I have done this at a customer site to overcome the 0.90.x slow WAL performance. With one RS per DN we bottlenecked, with 5-7 RS per DN we were able to hit the target rate.

Please note that we did this in lieu of the proper built-in options like WAL compression, multiple WAL, or n-way writes for WALs.

Lars

On Jun 16, 2012, at 10:53, Em <[EMAIL PROTECTED]> wrote:

> Stack,
>
> I have no issues with HBase, the question is purely theoretical.
>
>> So, you intend doubling the datanode instances per machine too?
> Everything else would not make sense to me, or what do you think?
>
> Thanks for your feedback!
>
> Regards,
> Em
>
> Am 16.06.2012 07:12, schrieb Stack:
>> On Fri, Jun 15, 2012 at 10:50 AM, Em <[EMAIL PROTECTED]> wrote:
>>> what are your experience with having more than one HBase-instance per
>>> machine?
>>>
>>
>> My experience is ittle.
>>
>> Its been done though by others on this list.
>>
>>> One reason to do this could be that you got i.e. 32 GB RAM and want to
>>> give 12-14 GB of RAM to each HBase instance so that you got little
>>> issues with Garbage Collection while using the available RAM-capacity
>>> for caching.
>>>
>>
>> You should leave some RAM for the file system cache.
>>
>> Are you unable to tame your GCs?  Is that why the two servers per node?
>>
>>> One thing I worry about is replication of data. Say two of three
>>> replicas reside at the same box in two distinct HBase instances.
>>
>> So, you intend doubling the datanode instances per machine too?
>>
>> St.Ack
>>