-Re: What exactly happens if fetch size is smaller than the next batch (0.72 and high-level consumer)
Philip O'Toole 2013-06-02, 01:26
Neha -- thanks very much. I had been thinking about transmitted batch
sizes, but of course, the Message Set header is stripped off the batch
by Kafka upon reception, and just the messages sit on disk. It is the
size of those messages that matters.
But that stall can happen -- thanks.
On Sat, Jun 1, 2013 at 1:25 PM, Neha Narkhede <[EMAIL PROTECTED]> wrote:
> Yes, it is the maxSize parameter in FetchRequest. And the consumer won't
> stall unless a single message (compressed or not) is larger than the fetch
> size, in this case 300KB. It doesn't matter how big a batch is.
> On Fri, May 31, 2013 at 11:25 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
>> Hello -- I'll try to look at the code, but I'm seeing something here
>> and I want to be *sure* I'm correct.
>> Say a batch sitting in a 0.72 partition is, say, 5MB in size. An
>> instance of a high-level consumer has a configured fetch size of
>> 300KB. This actually becomes the "maxSize" value, right, in
>> FetchRequest? So in this example does the high-level consumer stall?