Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HBase becomes ultra-slaggy


+
Asaf Mesika 2012-07-09, 18:35
+
Stack 2012-07-09, 19:19
Copy link to this message
-
Re: HBase becomes ultra-slaggy
I got JMX counters hooked up to JConsole (couple of them opened).
Do you have any advice from your experience on what metrics I should focus on  to spot this issue?

On Jul 9, 2012, at 22:19 PM, Stack wrote:

> On Mon, Jul 9, 2012 at 8:35 PM, Asaf Mesika <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> My cluster started being incredibly slow in the past 2 days.
>> I've seen many Blocking updates on the region server logs, which lead me to believe HDFS creates is the bottleneck.
>> I ran a small test (hadoop fs -copyFromLocal big3_3Giga.tz.gz /tmp) which copies a 3.3G file, and I was surprised to see a rate of 11MB/sec, when it was previously around 28MB/sec.
>>
>> I have 3 RS computers, hosting DataNode and embedded zookeeper as well.
>> And 1 Master computer, hosting NameNode as well.
>>
>> The slowest computer has Core 2 Duo 2.93GHz, 4GB.
>>
>> What steps can I take to find out the source of this slowdown?
>
> Install ganglia or opentsdb and figure what your cluster is up to when its slow.
>
> St.Ack
+
Otis Gospodnetic 2012-07-10, 03:54
+
Asaf Mesika 2012-07-10, 04:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB