For me scalability is to achieve same throughput and latency with the
increase in number of clients.
In my case the data set increases with the number of clients. That's the
reason I vary both clients and region servers.
I'm trying to identify how the cluster should grow to handle data from more
clients so that the operations throughput and latency is under defined
Currently the limit is 15K OPS throughput and 1 ms latency.
To test, I have kept the data increase at around 15 million per server.
Each YCSB client actually runs 32 threads. So it is actually 15 million
more data for 32 more clients.
All machines are physical servers.
1) Read and write latency is around 1 ms in first whereas in second case
its little higher at 1.1 to 1.2 ms.
2) Keeping same number of clients as the first case, the latency reduced to
0.7 ms but throughput came down further to just 9K OPS
For the tests, I'm running both clients and Region servers on same machine.
But I tried in 8 Server scenario to run clients on different machines but
results were almost same as that of running clients on same machine.
Ganglia shows that system load is around 30% in both scenarios.
What I wanted to understand is how to grow the cluster to meet the needs of
both throughput and latency?