Kafka, mail # user - Re: implicit default minimum retention size per partition is 4GB. - 2013-09-10, 16:50
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: implicit default minimum retention size per partition is 4GB.
You at least have to retain data long enough for the followers to be able
to copy it from the leader. As Jun mentioned, you can observe the max lag
in the
follower and set replica.lag.max.messages to be a little higher than that.
The high watermark is incremented to point to the logical offset of the
latest message that has been copied to all the replicas in the ISR. Once
the high watermark includes a certain message, it is considered
"committed". So it is not configurable.

Thanks,
Neha
On Tue, Sep 10, 2013 at 8:26 AM, Yu, Libo <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB