Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> changing computation at runtime


Copy link to this message
-
Re: changing computation at runtime
Ben,

For 1), yes.

For 2) and 3), you can shutdown your consumer cleanly, do the code upgrade
and start the consumer again. On restart, Kafka will make sure that the
consumer picks up data from where it's left off and there should be no data
loss.

Thanks,

Jun

On Thu, Jan 3, 2013 at 7:07 AM, "Benjamin Süß" <[EMAIL PROTECTED]> wrote:

> Hi there,
>
> I am evaluating several messaging platforms for using them in combination
> with a CEP engine. This targets developing a system for some kind of
> distributed CEP. I already have built a prototype doing some very basic
> analysis using Kafka & Esper.
>
> Now I am wondering about runtime modification possibilities of this system:
>
> 1.) From what I've read and done so far, I suppose I can just add new
> calculations  by registering new Consumers to new topics, is this correct?
>
> 2.) If so, how about updating existing calculations/applications? Will
> unregistering the old consumer and registering a new consumer to the same
> topic do the trick?
>
> 3.) Given questions 1&2 will be answered with yes, is there any way to
> prevent data loss while switching between versions?
>
> In case I just did not find the appropriate part of the documentation,
> just give me a hint/link. Any help will be appreciated.
>
> Regards,
> Benjamin
>