Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Decreasing write speed over time


Copy link to this message
-
RE: Decreasing write speed over time
Ramkrishna.S.Vasudevan 2012-06-19, 14:16
Hi

 

What about the region size and how frequently flush is happening?  Are you using default configurations only?

 

Regards

Ram

 

From: Giorgi Jvaridze [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, June 19, 2012 7:26 PM
To: [EMAIL PROTECTED]
Subject: Re: Decreasing write speed over time

 

Hi thanks for answer,

 

I've attached image of the chart.

Write performance is going down by the time. and never goes up.

As you can see on chart there are spikes but beside that generally write performance goes down over time.

 

Thanks.

Giorgi

On Tue, Jun 19, 2012 at 3:21 PM, Anoop Sam John <[EMAIL PROTECTED]> wrote:

Hi
  Sorry I was not able to access your chart of put perf. Is it like the performance gradually coming down and at some point again getting better all of a sudden?  [some spikes in graph]
It may be because of HBASE-3484

You can check the memstore size that u have given and see whether the spikes coming at the memstore flush time and then again perf going down gradually...

-Anoop-
__________
From: Giorgi Jvaridze [[EMAIL PROTECTED]]
Sent: Monday, June 18, 2012 7:28 PM
To: [EMAIL PROTECTED]
Subject: Decreasing write speed over time
Hello list,

I have Hbase (CDH4) setup of a master and 4 region servers (8 core 2.0GHz,
6GB of RAM and 6 100GB disks per data node). All of them are virtual
machines.

I'm writing into a table using single thread w/ auto flush off.
Ids are increased by one and then reversed. Data is a string with average
length 150 chars.

I'm measuring time needed for writing one million rows. At first I got
quite good results, it could write about 40-50K rows in a second but after
time this speed decreased.
I have a chart showing how It has been decreased by the time and data got
bigger. At the end of chart it's about 2 billion rows inserted.

http://i.minus.com/iRFtEoVJVsFso.png

Is this expected behavior for this kind of setup or something could went
wrong and I should check it up?

Thanks!

Giorgi