Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Wrong RPS stats?


Copy link to this message
-
Re: Wrong RPS stats?
bq. The client tells me 480 RPS

Were you using ycsb as client ?

On Mon, Jul 29, 2013 at 2:49 PM, Vladimir Rodionov
<[EMAIL PROTECTED]>wrote:

> Nope, I am doing simple table.get(get) - not a batch request
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: lars hofhansl [[EMAIL PROTECTED]]
> Sent: Monday, July 29, 2013 2:39 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Wrong RPS stats?
>
> Are you bulk requests put(List<Put>), etc? If so you get X RPCs and X*N
> requests if the list has N items on average.
> I admit this is a bit confusing.
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Vladimir Rodionov <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Cc:
> Sent: Monday, July 29, 2013 2:36 PM
> Subject: Wrong RPS stats?
>
> I am running simple performance test to evaluate maximum read IO we can
> get from 1 RS. The client tells me 480 RPS,
> the RS stats (from HBase UI) gives me totally different number:
>
> xxx.xxx.com,60020,1375123451578    Mon Jul 29 18:44:11 GMT+00:00 2013
>  requestsPerSecond=11684, numberOfOnlineRegions=398, usedHeapMB=70,
> maxHeapMB=5987
>
> 11684 RPS.
>
> HBase 0.94.6.1 (CDH4.3)
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: Vladimir Rodionov
> Sent: Monday, July 29, 2013 10:26 AM
> To: [EMAIL PROTECTED]; lars hofhansl
> Subject: RE: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> We have 20 RS and table was initially pre-split into 20 regions.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: lars hofhansl [[EMAIL PROTECTED]]
> Sent: Sunday, July 28, 2013 10:39 PM
> To: [EMAIL PROTECTED]
> Subject: Re: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> It starts with smaller regions and then ramps up the size as the table in
> question is growing in size.
> Makes make for a more even distribution of regions.
>
> Into how many region did you presplit your table? How many region servers
> do you have available in this cluster?
>
> Maybe that number was less than what that policy estimated to be a good
> number of regions for your cluster.
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Vladimir Rodionov <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Cc:
> Sent: Sunday, July 28, 2013 10:25 PM
> Subject: RE: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> Thanks, Ted. Are there  any rationals behind
>  IncreasingToUpperBoundRegionSplitPolicy ?
> Why is it better than ConstantSizeRegionSplitPolicy?
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: Ted Yu [[EMAIL PROTECTED]]
> Sent: Sunday, July 28, 2013 8:52 PM
> To: [EMAIL PROTECTED]
> Subject: Re: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> In 0.94 there're several split policies available:
>
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/ConstantSizeRegionSplitPolicy.java
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/DelimitedKeyPrefixRegionSplitPolicy.java
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/DisabledRegionSplitPolicy.java
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/IncreasingToUpperBoundRegionSplitPolicy.java
>
> ./src/main/java/org/apache/hadoop/hbase/regionserver/KeyPrefixRegionSplitPolicy.java
>
> From
> ./src/main/java/org/apache/hadoop/hbase/regionserver/RegionSplitPolicy.java
> :
>
> * @see IncreasingToUpperBoundRegionSplitPolicy Default split policy since
> *      0.94.0
> * @see ConstantSizeRegionSplitPolicy Default split policy before 0.94.0