Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Multiple region servers per physical node


Copy link to this message
-
Re: Multiple region servers per physical node
Its possible to walk to the middle of the Golden Gate Bridge, evade security and take a swan dive in to the Bay.
If you're in NY, substitute Golden Gate with Brooklyn ....

The point is that while something is possible, doesn't mean that its a good idea to do it.

Elliot makes this point...
>  It's a head-ache to run and administer, but if you have
> machines with lots of spindles it's an option.  Though it's not the
> recommended way to run HBase so there may be scary edges.

The key phrase ... "Its a head-ache to run and administer"
In the real world... there are things like BOFHs and its best not to get on their bad side. ;-)

But there are other options...
If you do have too man spindles, too much memory and too many cores.... you have another option...
Virtualize your nodes. You will take a performance hit on disk and network I/O but it gets balanced out.
VMWare (err Pivotal) has made claims about improving their virtualization software so that there is less overhead. Note that YMMV.

This way you can run more RS on the existing hardware. Note that you don't run HBase alone and  with the loss of memory and cores, you would end up reducing the number of slots per server.

But its a viable solution and its recommended over trying to run multiple RS on a node.

HTH

-Mike
On Jul 29, 2013, at 5:14 PM, Elliott Clark <[EMAIL PROTECTED]> wrote:

> Currently it's very possible to run multiple region servers per
> machine.  People who have benchmarked it have even found that it's
> faster (there are more wal's being used in addition to the extra
> heap).  It's a head-ache to run and administer, but if you have
> machines with lots of spindles it's an option.  Though it's not the
> recommended way to run HBase so there may be scary edges.
>
> Yes the G1 looks promising if you have a very read heavy workload
> (We've been running it for integration tests for about a month or so).
> But until multiple wal's are in HBase (it's going to be a while) some
> users might want try other solutions.
>
>
> On Mon, Jul 29, 2013 at 2:22 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> Have you seen this thread ?
>>
>> http://search-hadoop.com/m/PhUxhdw5Mz1/otis+g1gc&subj=Re+G1+before+after+GC+time+graph
>>
>> On Mon, Jul 29, 2013 at 2:19 PM, Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN) <
>> [EMAIL PROTECTED]> wrote:
>>
>>> If someone was contemplating running multiple HBase region servers per
>>> node, what would you tell him? I've lots of RAM on my machines and I want
>>> HBase to be able to make the most of it. Heaps larger than 12-16G have
>>> significant gc penalties. I could try to have a larger cache off-heap, but
>>> that feature is considered to be not fully baked in yet (according to
>>> people in the know). The option left over is to run multiple region servers
>>> per node. What are the downsides to that?
>>>
>>> Regards,
>>> -sudarshan
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB