Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Decreasing write speed over time


+
Giorgi Jvaridze 2012-06-18, 13:58
Copy link to this message
-
RE: Decreasing write speed over time
Hi
   Sorry I was not able to access your chart of put perf. Is it like the performance gradually coming down and at some point again getting better all of a sudden?  [some spikes in graph]
It may be because of HBASE-3484

You can check the memstore size that u have given and see whether the spikes coming at the memstore flush time and then again perf going down gradually...

-Anoop-
__________
From: Giorgi Jvaridze [[EMAIL PROTECTED]]
Sent: Monday, June 18, 2012 7:28 PM
To: [EMAIL PROTECTED]
Subject: Decreasing write speed over time

Hello list,

I have Hbase (CDH4) setup of a master and 4 region servers (8 core 2.0GHz,
6GB of RAM and 6 100GB disks per data node). All of them are virtual
machines.

I'm writing into a table using single thread w/ auto flush off.
Ids are increased by one and then reversed. Data is a string with average
length 150 chars.

I'm measuring time needed for writing one million rows. At first I got
quite good results, it could write about 40-50K rows in a second but after
time this speed decreased.
I have a chart showing how It has been decreased by the time and data got
bigger. At the end of chart it's about 2 billion rows inserted.

http://i.minus.com/iRFtEoVJVsFso.png

Is this expected behavior for this kind of setup or something could went
wrong and I should check it up?

Thanks!

Giorgi
+
Giorgi Jvaridze 2012-06-19, 13:56
+
Ramkrishna.S.Vasudevan 2012-06-19, 14:22
+
Ramkrishna.S.Vasudevan 2012-06-19, 14:16
+
Giorgi Jvaridze 2012-06-19, 15:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB