Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - batch sending in sync mode


+
Jason Rosenberg 2013-06-24, 08:29
+
Joel Koshy 2013-06-24, 08:59
+
Jason Rosenberg 2013-06-24, 09:29
Copy link to this message
-
Re: batch sending in sync mode
Sriram Subramanian 2013-06-24, 09:24
The messages will be grouped by their destination broker and further
grouped by topic/partition. The send then happens to each broker with a
list of topic/partitions and messages for them and waits for an
acknowledgement from each broker. This happens sequentially. So, the
messages are acknowledged in batches per broker. Going forward, we are
looking into providing some kind of callback mechanism for the async mode
after a successful send that would help one to perform further processing.

On 6/24/13 1:29 AM, "Jason Rosenberg" <[EMAIL PROTECTED]> wrote:

>I have been using async mode with 0.7.2, but I'm wondering if I should
>switch to sync mode, so I can use the new request.required.acks mode in a
>sensible way.
>
>I am already managing an async queue that then dispatches to the samsa
>producer.
>
>I'm wondering how the acknowledgement mode works when sending lists of
>messages with a synchronous producer.  Will each message in the list be
>individually acknowledged, or is there a chance for it to be acknowledged
>as a single batch (which would appear to be more desirable for efficiency
>reasons)?
>
>Thanks,
>
>Jason