Kafka, mail # user - Re: producer creating multiple log files after message sending - 2013-04-18, 16:20
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: producer creating multiple log files after message sending
I think there is a misunderstanding here. The log file division is
completely transparent to the consumer. The consumer consumes by
offset (0,1,2,3,...) and these offsets span log files (i.e. there is a
single, monotonically increasing set of offsets over all log files in
a partition). In other words consumers ALWAYS consume across log files
based on the position (offset) in the set of all logs in the
partition.

If you want fewer log files change the settings so that the log files
are bigger. However the only impact of this is fewer file handles, it
won't impact the consumer in any way at all.

Not sure if that helps...
-Jay

On Thu, Apr 18, 2013 at 8:36 AM, Jason Huang <[EMAIL PROTECTED]> wrote:

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB