Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> How well does HBase run on low/medium memory/cpu clusters?


+
David Parks 2012-10-10, 08:16
+
Michael Segel 2012-10-10, 13:51
+
David Parks 2012-10-11, 07:04
+
Anoop Sam John 2012-10-11, 08:10
Copy link to this message
-
Re: How well does HBase run on low/medium memory/cpu clusters?
Hi David,

For productions, it will be recommanded to have good computers. But
for testing, you can do with almost anything.

I'm running cluster in production with 6 region servers. My smallest
computer is a P4 with only 500M of memory... I'm not saying that it's
recommanded but it's working.

I can't use big cache values else I'm getting out of memory and
servers are closing (but not hadoop). But it's working. I have few
tables. My biggest one is 15M but is growing every day. I have
splitted it a lot to make sure to share the workload between all the
servers.

I'm expecting to be at about 30M by the end of the week.

The only big computer I have is my master (8CPU, 12G) wich is also
hosting a region server (not recommended for quality production
schema).

JM

2012/10/10, David Parks <[EMAIL PROTECTED]>:
> In looking at the AWS MapReduce version of HBase, it doesn't  even  give an
> option to run it on lower end hardware.
>
> I am considering HBase as an alternative to one large table we have in
> MySQL
> which is causing problems. It's 50M rows, a pretty straight forward set of
> product  items.
>
> The challenge  is that I need to do 10+ range scans a day over about 7M
> items each where we check for updates. This is ideal for HBase, but hell
> for
> MySQL (a join of a 7M row table with a 50M row table is giving us
> fits-a-plenty).
>
> But beyond the daily range scans the actual workload on the boxes should be
> reasonable, just random access reads. So it doesn't seem like I should need
> significant memory/CPU requirements...
>
> But here's where I don't find a lot of information - as  someone reasonably
> new to HBase (I read a book, did the examples), am I missing anything in my
> thinking?
>
> David
>
>