Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Performance of jBOD


Copy link to this message
-
Re: Performance of jBOD
Graeme Wallace 2013-10-03, 18:08
Jun,

Just to follow up - have you any stats on what disk read + write throughput
you are getting in production at LinkedIn ?

regards,

Graeme

On Thu, Oct 3, 2013 at 7:17 AM, Graeme Wallace <
[EMAIL PROTECTED]> wrote:

> Its whatever dstat reports - i think it must be MBytes ?
>
>
> On Wed, Oct 2, 2013 at 10:29 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
>> If you use a reasonable fetch size in the consumer (e.g. 100+KB) or above,
>> you could probably get most of the sequential scan performance from those
>> disks. Do you mean 250 Mbits or MBytes?
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Wed, Oct 2, 2013 at 4:35 PM, Graeme Wallace <
>> [EMAIL PROTECTED]> wrote:
>>
>> > We notice that we get a lot higher performance from Kafka up until the
>> > point where disk reads are taking place. (which is to be expected as its
>> > reading from filesystem cache).
>> >
>> > However, given that we have 12x7200rpm SATA disks configured as jBOD -
>> what
>> > sort of disk read performance should i see reported by dstat when things
>> > are running flat out ?
>> >
>> > I'm seeing 250-340Mb sustained.
>> >
>> > --
>> > Graeme Wallace
>> > CTO
>> > FareCompare.com
>> > O: 972 588 1414
>> > M: 214 681 9018
>> >
>>
>
>
>
> --
> Graeme Wallace
> CTO
> FareCompare.com
> O: 972 588 1414
> M: 214 681 9018
>
>
--
Graeme Wallace
CTO
FareCompare.com
O: 972 588 1414
M: 214 681 9018