Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Flume hops behind HAProxy


+
Rahul Ravindran 2012-11-14, 01:52
+
Rahul Ravindran 2012-11-14, 20:46
+
Brock Noland 2012-11-14, 21:33
+
Rahul Ravindran 2012-11-14, 21:49
+
Camp, Roy 2012-11-14, 22:15
Copy link to this message
-
Re: Flume hops behind HAProxy
HAProxy is widely deployed already in our deployment and Ops is familiar with dealing with it for hosts which go down etc. 
________________________________
 From: "Camp, Roy" <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Wednesday, November 14, 2012 2:15 PM
Subject: RE: Flume hops behind HAProxy
 
Out of curiosity, what is the use case vs using the built in load balancing?

-----Original Message-----
From: Rahul Ravindran [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 14, 2012 1:49 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Flume hops behind HAProxy

It would be round robin but not sticky sessions( so each request could Goto any random flume hop)

Sent from my phone.Excuse the terseness.

On Nov 14, 2012, at 1:33 PM, Brock Noland <[EMAIL PROTECTED]> wrote:

> I assume it would be connection based round robin?  Might work just
> fine, but probably best to the use built-in support.
>
> On Wed, Nov 14, 2012 at 2:46 PM, Rahul Ravindran <[EMAIL PROTECTED]> wrote:
>> Resending given I sent it during off-hours.
>>
>> ________________________________
>> From: Rahul Ravindran <[EMAIL PROTECTED]>
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Sent: Tuesday, November 13, 2012 5:52 PM
>> Subject: Flume hops behind HAProxy
>>
>> Hi,
>>   Before I try it, I wanted to check if there were any known issues
>> with this. We will have multiple flume agents sending an Avro stream
>> each to a smaller set of intermediate flume hops. Are there any
>> issues/concerns around having the flume agents send their streams to
>> an HAProxy which will round robin between the different flume hops.
>> Any issue around the transaction mechanism with this setup?
>>
>> I know that there is a selector mechanism in Flume to do this, but
>> our operations extensively use HAProxy, and are most familiar with it.
>>
>> Thanks,
>> ~Rahul.
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
+
Brock Noland 2012-11-15, 02:20
+
Rahul Ravindran 2012-11-15, 17:27
+
Brock Noland 2012-11-15, 17:35