Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Multiple HBase instances per machine


+
Em 2012-06-15, 17:50
+
Stack 2012-06-16, 05:12
+
Em 2012-06-16, 08:53
Copy link to this message
-
Re: Multiple HBase instances per machine
Hi,

I have done this at a customer site to overcome the 0.90.x slow WAL performance. With one RS per DN we bottlenecked, with 5-7 RS per DN we were able to hit the target rate.

Please note that we did this in lieu of the proper built-in options like WAL compression, multiple WAL, or n-way writes for WALs.

Lars

On Jun 16, 2012, at 10:53, Em <[EMAIL PROTECTED]> wrote:

> Stack,
>
> I have no issues with HBase, the question is purely theoretical.
>
>> So, you intend doubling the datanode instances per machine too?
> Everything else would not make sense to me, or what do you think?
>
> Thanks for your feedback!
>
> Regards,
> Em
>
> Am 16.06.2012 07:12, schrieb Stack:
>> On Fri, Jun 15, 2012 at 10:50 AM, Em <[EMAIL PROTECTED]> wrote:
>>> what are your experience with having more than one HBase-instance per
>>> machine?
>>>
>>
>> My experience is ittle.
>>
>> Its been done though by others on this list.
>>
>>> One reason to do this could be that you got i.e. 32 GB RAM and want to
>>> give 12-14 GB of RAM to each HBase instance so that you got little
>>> issues with Garbage Collection while using the available RAM-capacity
>>> for caching.
>>>
>>
>> You should leave some RAM for the file system cache.
>>
>> Are you unable to tame your GCs?  Is that why the two servers per node?
>>
>>> One thing I worry about is replication of data. Say two of three
>>> replicas reside at the same box in two distinct HBase instances.
>>
>> So, you intend doubling the datanode instances per machine too?
>>
>> St.Ack
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB