Kafka, mail # user - Re: producer creating multiple log files after message sending - 2013-04-18, 16:20
 Search Hadoop and all its subprojects:

Switch to Plain View
Snehalata Nagaje 2013-04-17, 07:06
Neha Narkhede 2013-04-17, 15:38
Snehalata Nagaje 2013-04-18, 06:11
Snehalata Nagaje 2013-04-18, 08:53
Swapnil Ghike 2013-04-18, 09:43
Jason Huang 2013-04-18, 13:07
Neha Narkhede 2013-04-18, 15:26
Jason Huang 2013-04-18, 15:37
Copy link to this message
Re: producer creating multiple log files after message sending
I think there is a misunderstanding here. The log file division is
completely transparent to the consumer. The consumer consumes by
offset (0,1,2,3,...) and these offsets span log files (i.e. there is a
single, monotonically increasing set of offsets over all log files in
a partition). In other words consumers ALWAYS consume across log files
based on the position (offset) in the set of all logs in the

If you want fewer log files change the settings so that the log files
are bigger. However the only impact of this is fewer file handles, it
won't impact the consumer in any way at all.

Not sure if that helps...

On Thu, Apr 18, 2013 at 8:36 AM, Jason Huang <[EMAIL PROTECTED]> wrote:

Jason Huang 2013-04-18, 16:36
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB