Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Graceful termination of kafka broker after draining all the data consumed

Copy link to this message
Re: Graceful termination of kafka broker after draining all the data consumed
0.8 sounds really great!

OK, I will try after you release stable build of 0.8

Thank you
Best, Jae

On Sun, Jan 6, 2013 at 10:36 AM, Neha Narkhede <[EMAIL PROTECTED]> wrote:
> In 0.8, we will provide a way for your to shutdown the broker in a
> controlled fashion. What that would include is moving all the leaders away
> from the broker so that it does not take any more produce requests. Once
> that is done, you can shutdown the broker normally. You don't have to wait
> until the consumers have pulled the data simply because they can consume
> from the new leader.
> Thanks,
> Neha
> On Sun, Jan 6, 2013 at 8:52 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>> In 0.7, one way to do this is to use a vip. All producers send data to the
>> vip. To decommission a broker,  you first take the broker out of vip so no
>> new data will be produced to it. Then you let the consumer drain the data
>> (you can use ConsumerOffsetChecker to check if all data has been consumed).
>> Finally, you can shut down the broker.
>> This will be much easier in 0.8 because of replication.
>> Thanks,
>> Jun
>> On Sat, Jan 5, 2013 at 11:34 PM, Bae, Jae Hyeon <[EMAIL PROTECTED]>
>> wrote:
>> > Hi
>> >
>> > If I want to terminate kafka broker gracefully. Before termination, it
>> > should stop receiving the traffic from producers and wait until all
>> > data will be consumed.
>> >
>> > I don't think that kafka 0.7.x is supporting this feature. If I want
>> > to implement this feature for myself, could you give me a brief sketch
>> > of implementation?
>> >
>> > Thank you
>> > Best, Jae
>> >