Asaf - check the URL in my signature - it's a service for monitoring various HBase metrics and may visually show you what is going on with various parts of your 3 servers.
Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm
----- Original Message -----
> From: Asaf Mesika <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Monday, July 9, 2012 2:35 PM
> Subject: HBase becomes ultra-slaggy
> My cluster started being incredibly slow in the past 2 days.
> I've seen many Blocking updates on the region server logs, which lead me to
> believe HDFS creates is the bottleneck.
> I ran a small test (hadoop fs -copyFromLocal big3_3Giga.tz.gz /tmp) which copies
> a 3.3G file, and I was surprised to see a rate of 11MB/sec, when it was
> previously around 28MB/sec.
> I have 3 RS computers, hosting DataNode and embedded zookeeper as well.
> And 1 Master computer, hosting NameNode as well.
> The slowest computer has Core 2 Duo 2.93GHz, 4GB.
> What steps can I take to find out the source of this slowdown?
> Asaf Mesika