Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Wrong RPS stats?


Copy link to this message
-
Re: Wrong RPS stats?
bq. The client tells me 480 RPS

Were you using ycsb as client ?

On Mon, Jul 29, 2013 at 2:49 PM, Vladimir Rodionov
<[EMAIL PROTECTED]>wrote:

> Nope, I am doing simple table.get(get) - not a batch request
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: lars hofhansl [[EMAIL PROTECTED]]
> Sent: Monday, July 29, 2013 2:39 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Wrong RPS stats?
>
> Are you bulk requests put(List<Put>), etc? If so you get X RPCs and X*N
> requests if the list has N items on average.
> I admit this is a bit confusing.
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Vladimir Rodionov <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Cc:
> Sent: Monday, July 29, 2013 2:36 PM
> Subject: Wrong RPS stats?
>
> I am running simple performance test to evaluate maximum read IO we can
> get from 1 RS. The client tells me 480 RPS,
> the RS stats (from HBase UI) gives me totally different number:
>
> xxx.xxx.com,60020,1375123451578    Mon Jul 29 18:44:11 GMT+00:00 2013
>  requestsPerSecond=11684, numberOfOnlineRegions=398, usedHeapMB=70,
> maxHeapMB=5987
>
> 11684 RPS.
>
> HBase 0.94.6.1 (CDH4.3)
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: Vladimir Rodionov
> Sent: Monday, July 29, 2013 10:26 AM
> To: [EMAIL PROTECTED]; lars hofhansl
> Subject: RE: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> We have 20 RS and table was initially pre-split into 20 regions.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: lars hofhansl [[EMAIL PROTECTED]]
> Sent: Sunday, July 28, 2013 10:39 PM
> To: [EMAIL PROTECTED]
> Subject: Re: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> It starts with smaller regions and then ramps up the size as the table in
> question is growing in size.
> Makes make for a more even distribution of regions.
>
> Into how many region did you presplit your table? How many region servers
> do you have available in this cluster?
>
> Maybe that number was less than what that policy estimated to be a good
> number of regions for your cluster.
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Vladimir Rodionov <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Cc:
> Sent: Sunday, July 28, 2013 10:25 PM
> Subject: RE: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> Thanks, Ted. Are there  any rationals behind
>  IncreasingToUpperBoundRegionSplitPolicy ?
> Why is it better than ConstantSizeRegionSplitPolicy?
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: Ted Yu [[EMAIL PROTECTED]]
> Sent: Sunday, July 28, 2013 8:52 PM
> To: [EMAIL PROTECTED]
> Subject: Re: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> In 0.94 there're several split policies available:
>
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitPolicy.java
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/DisabledRegionSplitPolicy.java
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.java
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitPolicy.java
>
> From
> ./src/main/java/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.java
> :
>
> * @see IncreasingToUpperBoundRegionSplitPolicy Default split policy since
> *      0.94.0
> * @see ConstantSizeRegionSplitPolicy Default split policy before 0.94.0
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB