Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> who is using kafka to stare large messages?


Copy link to this message
-
Re: who is using kafka to stare large messages?
I see, so that is one thing to consider is if I have 20 KB messages, I
shouldn't batch too many together as that will increase latency and the
memory usage footprint on the producer side of things.
On Mon, Oct 7, 2013 at 11:55 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> At LinkedIn, our message size can be 10s of KB. This is mostly because we
> batch a set of messages and send them as a single compressed message.
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 7, 2013 at 7:44 AM, S Ahmed <[EMAIL PROTECTED]> wrote:
>
> > When people using message queues, the message size is usually pretty
> small.
> >
> > I want to know who out there is using kafka with larger payload sizes?
> >
> > In the configuration, the maximum message size by default is set to 1
> > megabyte (
> > message.max.bytes1000000)
> >
> > My message sizes will be probably be around 20-50 KB but to me that is
> > large for a message payload so I'm wondering what effects that will have
> > with kafka.
> >
>