Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> 0.7 -> 0.8 Protocol Upgrade in production environments


Copy link to this message
-
Re: 0.7 -> 0.8 Protocol Upgrade in production environments
In my case, we just rolled out a separate 0.8 cluster, and migrated
producers to it over time (took several weeks to get everything
updated to the new cluster).  In the transition, we had consumers
running for both clusters.  Once no traffic was flowing on the old
cluster, we then shut down the 0.7 cluster.

I think this was simpler than trying to forward data from one to the other.

You can't have the 2 existing in a mixed state in the same cluster, at
all.  The protocols and storage formats are entirely different.

Jason

On Tue, Jan 21, 2014 at 5:38 PM, François Langelier
<[EMAIL PROTECTED]> wrote:
> Hi,
>
> I'm also in the process to upgrade from 0.7 to 0.8. WIth the informations I
> found, your best friend is here :
> https://cwiki.apache.org/confluence/display/KAFKA/Migrating+from+0.7+to+0.8
> It's a migration tool from 0.7 to 0.8.
>
> You will have to install 0.8 while you still have you 0.7 running. When
> those and your 0.8 consumers run, you will have to lunch the migration tool
> that will consume your 0.7 message to produce them to your 0.8 environment.
> When all your message will be transfer to your 0.8, you will them be able
> to transfer your 0.7 producers to your 0.8 environment.
>
> A picture is worth 1000 words :
> [image: Inline image 1]
>
> So for you b) , yes it's possible, but as far as I know, you will need to
> have a tool to verify the duplication of your message because some message
> will probably be consume in 0.7 and 0.8 while you are migrating because the
> offsets are incompatible between 0.7 and 0.8...
>
> c) yes if you use the migration tool
>
>
>
>
>
> François Langelier
> Étudiant en génie Logiciel - École de Technologie
> Supérieure<http://www.etsmtl.ca/>
> Capitaine Club Capra <http://capra.etsmtl.ca/>
> VP-Communication - CS Games <http://csgames.org> 2014
> Jeux de Génies <http://www.jdgets.com/> 2011 à 2014
> Argentier Fraternité du Piranhas <http://fraternitedupiranha.com/> 2012-2014
> Comité Organisateur Olympiades ÉTS 2012
> Compétition Québécoise d'Ingénierie 2012 - Compétition Senior
>
>
> On Tue, Jan 21, 2014 at 12:48 PM, Clark Breyman <[EMAIL PROTECTED]> wrote:
>
>> I'm curious what the recommended best practice is for migrating a
>> production environment with replication from 0.7 to 0.8 given the protocol
>> upgrade. Some specific questions I have are:
>>
>> a) Is it possible to mix 0.7 and 0.8 servers for a given partition during
>> the migration?
>>
>> b) If we can't mix server versions, how do we do a zero-downtime upgrade?
>>
>> c) Will 0.7 clients continue to communicate correctly on-the-wire with the
>> 0.8 server until they can be upgraded?
>>
>>
>> Any war stories of production upgrades would be helpful. Thanks in advance.
>> - Clark
>>