Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - batch sending in sync mode


+
Jason Rosenberg 2013-06-24, 08:29
+
Joel Koshy 2013-06-24, 08:59
Copy link to this message
-
Re: batch sending in sync mode
Jason Rosenberg 2013-06-24, 09:29
In async mode, I don't think I have any way of handling send failures or
even knowing about them.  In sync mode, I can handle exceptions, etc.,
after max retries have happened.

Make sense?

Jason

On Mon, Jun 24, 2013 at 1:59 AM, Joel Koshy <[EMAIL PROTECTED]> wrote:

> If you send a list of messages in sync mode, those messages will be
> partitioned (randomly by default) and the collated messages will be sent
> out in batches to each broker. i.e., the original batch may get split into
> smaller batches - each of those batches is acknowledged. Why not use async
> mode with required.acks set to 1 (or -1)?
>
>
> On Mon, Jun 24, 2013 at 1:29 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
> > I have been using async mode with 0.7.2, but I'm wondering if I should
> > switch to sync mode, so I can use the new request.required.acks mode in a
> > sensible way.
> >
> > I am already managing an async queue that then dispatches to the samsa
> > producer.
> >
> > I'm wondering how the acknowledgement mode works when sending lists of
> > messages with a synchronous producer.  Will each message in the list be
> > individually acknowledged, or is there a chance for it to be acknowledged
> > as a single batch (which would appear to be more desirable for efficiency
> > reasons)?
> >
> > Thanks,
> >
> > Jason
> >
>

 
+
Sriram Subramanian 2013-06-24, 09:24