Hi，
     I implement producer and consumer with java api. But I could not get the performance according to official document. The official document show that producer could reach to 250000 nMessages/s and consumer could reach to 550000 nMessages/s. However, it just reach to 80000 nMessages/s produced and 40000 nMessages/s consumered. The test is run with a producer, a consumer, and two brokers. Did anyone have tested the performance with java client? How can I get the performance as well as offical document.
Thanks!

 

Typically, the performance is controlled by how often data is flush to disk
in the broker. Try to increase log.flush.interval. See
http://incubator.apache.org/kafka/configuration.html for its meaning.

Thanks,

Jun

On Tue, Apr 17, 2012 at 5:28 AM, <[EMAIL PROTECTED]> wrote:

> Hi，
>     I implement producer and consumer with java api. But I could not get
> the performance according to official document. The official document show
> that producer could reach to 250000 nMessages/s and consumer could reach to
> 550000 nMessages/s. However, it just reach to 80000 nMessages/s produced
> and 40000 nMessages/s consumered. The test is run with a producer, a
> consumer, and two brokers. Did anyone have tested the performance with java
> client? How can I get the performance as well as offical document.
> Thanks!
>
>
>
>

what kind of hardware are you running the test on?

On Tue, Apr 17, 2012 at 8:28 AM, <[EMAIL PROTECTED]> wrote:

> Hi，
>     I implement producer and consumer with java api. But I could not get
> the performance according to official document. The official document show
> that producer could reach to 250000 nMessages/s and consumer could reach to
> 550000 nMessages/s. However, it just reach to 80000 nMessages/s produced
> and 40000 nMessages/s consumered. The test is run with a producer, a
> consumer, and two brokers. Did anyone have tested the performance with java
> client? How can I get the performance as well as offical document.
> Thanks!
>
>
>
>

The tests were run on Linux machines with SATA drives.

Thanks,

Jun

On Sun, Apr 29, 2012 at 6:02 PM, S Ahmed <[EMAIL PROTECTED]> wrote:

> what kind of hardware are you running the test on?
>
> On Tue, Apr 17, 2012 at 8:28 AM, <[EMAIL PROTECTED]> wrote:
>
> > Hi，
> >     I implement producer and consumer with java api. But I could not get
> > the performance according to official document. The official document
> show
> > that producer could reach to 250000 nMessages/s and consumer could reach
> to
> > 550000 nMessages/s. However, it just reach to 80000 nMessages/s produced
> > and 40000 nMessages/s consumered. The test is run with a producer, a
> > consumer, and two brokers. Did anyone have tested the performance with
> java
> > client? How can I get the performance as well as offical document.
> > Thanks!
> >
> >
> >
> >
>

My message sizes will be 20-100K in size, how can I tweak the benchmarks to
use a payload size in 20-100K range?

On Mon, Apr 30, 2012 at 1:15 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> The tests were run on Linux machines with SATA drives.
>
> Thanks,
>
> Jun
>
> On Sun, Apr 29, 2012 at 6:02 PM, S Ahmed <[EMAIL PROTECTED]> wrote:
>
> > what kind of hardware are you running the test on?
> >
> > On Tue, Apr 17, 2012 at 8:28 AM, <[EMAIL PROTECTED]> wrote:
> >
> > > Hi，
> > >     I implement producer and consumer with java api. But I could not
> get
> > > the performance according to official document. The official document
> > show
> > > that producer could reach to 250000 nMessages/s and consumer could
> reach
> > to
> > > 550000 nMessages/s. However, it just reach to 80000 nMessages/s
> produced
> > > and 40000 nMessages/s consumered. The test is run with a producer, a
> > > consumer, and two brokers. Did anyone have tested the performance with
> > java
> > > client? How can I get the performance as well as offical document.
> > > Thanks!
> > >
> > >
> > >
> > >
> >
>

If you use ProducerPerformance, the payload size is configurable.

Thanks,

Jun

On Mon, Apr 30, 2012 at 11:03 AM, S Ahmed <[EMAIL PROTECTED]> wrote:

> My message sizes will be 20-100K in size, how can I tweak the benchmarks to
> use a payload size in 20-100K range?
>
> On Mon, Apr 30, 2012 at 1:15 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > The tests were run on Linux machines with SATA drives.
> >
> > Thanks,
> >
> > Jun
> >
> > On Sun, Apr 29, 2012 at 6:02 PM, S Ahmed <[EMAIL PROTECTED]> wrote:
> >
> > > what kind of hardware are you running the test on?
> > >
> > > On Tue, Apr 17, 2012 at 8:28 AM, <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hi，
> > > >     I implement producer and consumer with java api. But I could not
> > get
> > > > the performance according to official document. The official document
> > > show
> > > > that producer could reach to 250000 nMessages/s and consumer could
> > reach
> > > to
> > > > 550000 nMessages/s. However, it just reach to 80000 nMessages/s
> > produced
> > > > and 40000 nMessages/s consumered. The test is run with a producer, a
> > > > consumer, and two brokers. Did anyone have tested the performance
> with
> > > java
> > > > client? How can I get the performance as well as offical document.
> > > > Thanks!
> > > >
> > > >
> > > >
> > > >
> > >
> >
>