Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Running multiple flume versions on the same box

Copy link to this message
Re: Running multiple flume versions on the same box
Absolutely. I don't expect to need to do this and I expect to be staging any changes prior to a production deployment. 

This was a question from our operations on how non-compatible upgrades would be handled, and I needed to get an idea on how we may handle it, if this scenario, however rare does come up.

Thanks for all the info.


 From: Mike Percy <[EMAIL PROTECTED]>
Cc: Rahul Ravindran <[EMAIL PROTECTED]>
Sent: Wednesday, November 21, 2012 2:24 PM
Subject: Re: Running multiple flume versions on the same box

There are no system level singletons or hard-coded file paths or ports if that is what you mean.

But in a production scenario, Flume should be resilient to failures since it will just buffer events in the channel at each agent. So why run simultaneous versions when doing minor version upgrades? (I can understand in an OG -> NG migration) If there is a problem just take it down and roll back; the rest of the system should be fine if you have done sufficient capacity planning (with channel sizes) and configuration to tolerate downtime - which I'd strongly recommend.

At the end of the day, it's always best to test new versions in staging any time you do a software upgrade, including with Flume.

Hope that helps.


On Wed, Nov 21, 2012 at 1:29 PM, Camp, Roy <[EMAIL PROTECTED]> wrote:

We did this when upgrading from 0.9x to FlumeNG 1.3-SNAPSHOT.  Used different ports and different logging/data directories.  Worked great.


>From:Rahul Ravindran [mailto:[EMAIL PROTECTED]]
>Sent: Wednesday, November 21, 2012 11:24 AM
>To: User-flume
>Subject: Running multiple flume versions on the same box

>  This is primarily to try and address a flume upgrade scenario in the case of any incompatible changes in future. I tried this with multiple processes of the same version, and it appeared to work. Are there any concerns on running multiple versions of flume on the same box (each with different agent configurations where there is no overlap of ports) ?