Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> What exactly happens if fetch size is smaller than the next batch (0.72 and high-level consumer)


Copy link to this message
-
Re: What exactly happens if fetch size is smaller than the next batch (0.72 and high-level consumer)
Yes, it is the maxSize parameter in FetchRequest. And the consumer won't
stall unless a single message (compressed or not) is larger than the fetch
size, in this case 300KB. It doesn't matter how big a batch is.

Thanks,
Neha

On Fri, May 31, 2013 at 11:25 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:

> Hello -- I'll try to look at the code, but I'm seeing something here
> and I want to be *sure* I'm correct.
>
> Say a batch sitting in a 0.72 partition is, say, 5MB in size. An
> instance of a high-level consumer has a configured fetch size of
> 300KB. This actually becomes the "maxSize" value, right, in
> FetchRequest? So in this example does the high-level consumer stall?
>
> Thanks,
>
> Philip
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB