-Re: Graceful termination of kafka broker after draining all the data consumed
0.8 sounds really great!
OK, I will try after you release stable build of 0.8
On Sun, Jan 6, 2013 at 10:36 AM, Neha Narkhede <[EMAIL PROTECTED]> wrote:
> In 0.8, we will provide a way for your to shutdown the broker in a
> controlled fashion. What that would include is moving all the leaders away
> from the broker so that it does not take any more produce requests. Once
> that is done, you can shutdown the broker normally. You don't have to wait
> until the consumers have pulled the data simply because they can consume
> from the new leader.
> On Sun, Jan 6, 2013 at 8:52 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>> In 0.7, one way to do this is to use a vip. All producers send data to the
>> vip. To decommission a broker, you first take the broker out of vip so no
>> new data will be produced to it. Then you let the consumer drain the data
>> (you can use ConsumerOffsetChecker to check if all data has been consumed).
>> Finally, you can shut down the broker.
>> This will be much easier in 0.8 because of replication.
>> On Sat, Jan 5, 2013 at 11:34 PM, Bae, Jae Hyeon <[EMAIL PROTECTED]>
>> > Hi
>> > If I want to terminate kafka broker gracefully. Before termination, it
>> > should stop receiving the traffic from producers and wait until all
>> > data will be consumed.
>> > I don't think that kafka 0.7.x is supporting this feature. If I want
>> > to implement this feature for myself, could you give me a brief sketch
>> > of implementation?
>> > Thank you
>> > Best, Jae