Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Performance degrades on moving from desktop to blade environment


Copy link to this message
-
Re: Performance degrades on moving from desktop to blade environment
Himanish, it hard to say without trend graphs.  Setup ganglia and get
fsreadlatancy, as well as thread count graphs to see what the issue
might be.

-Jack

On Thu, May 19, 2011 at 11:46 AM, Himanish Kushary <[EMAIL PROTECTED]> wrote:
> Hi,
>
> Could anybody suggest what may be the issue. I ran YCSB on both the
> development and production servers.
>
> The loading of data performs better on the production cluster but the 50%
> read-50% write workloada performs better on the development.The average
> latency for read shoots up to 30-40 ms on production, for development it is
> between 10-20 ms.This was while running with 10 threads maintaining 1000 tps
> using this command - [*java -cp build/ycsb.jar:db/hbase/conf:db/hbase/lib/*
> com.yahoo.ycsb.Client -t -db com.yahoo.ycsb.db.HBaseClient -P
> workloads/workloada -p columnfamily=data -p operationcount=1000000 -s
> -threads 10 -target 1000*]
>
> The clusters seems to perform similiarly using YCSB when the tps and
> operationcount is lowered to 500 and 100000 respectively.
>
> We ran our Map-Reduces on the two clusters (assuming that we will not reach
> 1000 tps or that much of operationcount from the map-reduce), but strangely
> the development cluster performed better.
>
> Any suggestions will be really helpful?
>
> Thanks
> Himanish
>
>
>
> On Mon, May 16, 2011 at 4:43 PM, Himanish Kushary <[EMAIL PROTECTED]>wrote:
>
>> *PRODUCTION SERVER CPU INFO*
>> processor : 0
>> vendor_id : AuthenticAMD
>> cpu family : 16
>> model : 9
>> model name : AMD Opteron(tm) Processor 6174
>> stepping : 1
>> cpu MHz : 2200.022
>> cache size : 512 KB
>> physical id : 1
>> siblings : 12
>> core id : 0
>> cpu cores : 12
>> apicid : 16
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 5
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
>> pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp
>> lm 3dnowext 3dnow constant_tsc nonstop_tsc pni cx16 popcnt lahf_lm
>> cmp_legacy svm extapic cr8_legacy altmovcr8 abm sse4a misalignsse
>> 3dnowprefetch osvw
>> bogomips : 4400.03
>> TLB size : 1024 4K pages
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 48 bits physical, 48 bits virtual
>> power management: ts ttp tm stc 100mhzsteps hwpstate [8]
>>
>>
>> *DEVELOPMENT SERVER CPU INFO*
>>
>> processor : 0
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 30
>> model name : Intel(R) Core(TM) i7 CPU       Q 740  @ 1.73GHz
>> stepping : 5
>> cpu MHz : 933.000
>> cache size : 6144 KB
>> physical id : 0
>> siblings : 8
>> core id : 0
>> cpu cores : 4
>> apicid : 0
>> initial apicid : 0
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 11
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
>> pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
>> constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf
>> pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2
>> popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
>> bogomips : 3457.61
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 36 bits physical, 48 bits virtual
>> power management:
>>
>>
>>
>> On Mon, May 16, 2011 at 4:26 PM, Jack Levin <[EMAIL PROTECTED]> wrote:
>>
>>> What is the clock rate of your CPUs (desktop vs blade)?
>>>
>>> -Jack
>>>
>>> On Mon, May 16, 2011 at 1:24 PM, Himanish Kushary <[EMAIL PROTECTED]>
>>> wrote:
>>> > Yes, it is only the HW that was changed . All the configurations are
>>> kept at
>>> > default from the cloudera installer.
>>> >
>>> > The regionserver logs semms ok.
>>> >
>>> > On Mon, May 16, 2011 at 3:20 PM, Jean-Daniel Cryans <
>>> [EMAIL PROTECTED]>wrote:
>>> >
>>> >> Ok I see... so the only thing that changed is the HW right? No
>>> >> upgrades to a new version? Also could it be possible that you changed
>>> >> some configs (or missed them)? BTW counting has a parameter for
>>> >> scanner caching, like you would write: count "myTable", CACHE = 1000