Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Performance degrades on moving from desktop to blade environment


Copy link to this message
-
Re: Performance degrades on moving from desktop to blade environment
Jack Levin 2011-05-16, 19:35
We had issues of moving into 32 core AMD box also.  The issue was
revolving around datanode getting slow after about 12 hours.  What you
need to do is check fsreadlatency_ave_time graph, if it appears spiky
then you have a problem with IO, next get a graph of "Runnable
Threads"  they should be flatlining, if they are spiking you might
have IO/Memory contention.  Run RAM tests head to head, we found our
32 Core ECC RAM would always underperform while using sysbench, by a
large margin.

-Jack

On Mon, May 16, 2011 at 10:33 AM, Himanish Kushary <[EMAIL PROTECTED]> wrote:
> Hi,
>
> We are in the process of moving a small Hbase/Hadoop cluster from our
> development to production environment.Our development environment were few
> intel desktops (8 cores CPU/8 Gigs RAM/7200 rpm disks) running centOS while
> the production environment are blades with (24 cores AMD CPU/32 gigs
> RAM/15000 rpm disks) AMD architecture running centOS.
>
> Strangely the hbase performance seems to degrade after moving stuffs to the
> production enviroment (suppoesed to have more horse power).We are using the
> latest and default installation for cloudera version of hadoop and hbase.No
> changes to memory or other parameter were done on both the environment.
>
> Any idea what could cause this.Could the AMD architecture be the
> cause.Pointers to things to look for to improve performance in the
> production cluster would be really appreciated.
>
> Note: We ran "count" from hbase shell on a huge table and found the desktops
> to be performing much better. We are in the process of comparing Map-Reduces
> presently.
>
> ---------------------------
> Thanks & Regards
> Himanish
>