Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Flume NG and zookeeper


+
吳瑞琳 2013-02-08, 02:36
+
Roshan Naik 2013-02-08, 02:41
+
Brock Noland 2013-02-08, 02:40
+
Mike Percy 2013-02-08, 02:41
+
吳瑞琳 2013-02-08, 03:07
+
Mike Percy 2013-02-08, 03:17
+
吳瑞琳 2013-02-08, 03:50
+
Tech Mail 2013-02-08, 14:44
Copy link to this message
-
Re: Flume NG and zookeeper
you could put flume behind a  VIP and not expose the actual flume nodes
names to clients.
-roshan
On Fri, Feb 8, 2013 at 6:44 AM, Tech Mail <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am looking for similar functionality, but we loose the flexibility.
>
> Cannot add a new agent dynamically to the list of collectors.
> Cannot move the agents from one host to other host or from one datacenter
> to other.
> Entire list of agents has to be provided to all end user applications.
>
> How to dynamically get the host/port of the agent from the client
> application?
>    LDAP server server.
>     Any other ways ?
>
> Yogi
>
> On Feb 7, 2013, at 7:50 PM, 吳瑞琳 <[EMAIL PROTECTED]> wrote:
>
> Mike,
>
> Thanks a lot. It fits our need. I'll try your advice.
>
> Regards,
> RL
>
> 2013/2/8 Mike Percy <[EMAIL PROTECTED]>
>
>> Interesting use case. Have you considered using the existing
>> polling-based load balancing RPC client functionality?
>>
>> http://flume.apache.org/FlumeDeveloperGuide.html#loadbalancing-rpc-client
>>
>> Regards,
>> Mike
>>
>>
>> On Thu, Feb 7, 2013 at 7:07 PM, 吳瑞琳 <[EMAIL PROTECTED]> wrote:
>>
>>> Thank you all.
>>>
>>> I am trying configure multiple hosts running Flume to collect data from
>>> clients. I want to use Zookeeper to track status of Flume hosts so that my
>>> clients can get the list of available Flume host and send data to one of
>>> them. I would like to make Flume hosts to register an ephemeral node under
>>> some Zookeeper node when it is up.
>>>
>>> Regards,
>>> RL
>>>
>>>
>>> 2013/2/8 Mike Percy <[EMAIL PROTECTED]>
>>>
>>>> Integrate in what way?
>>>>
>>>>
>>>> On Thu, Feb 7, 2013 at 6:36 PM, 吳瑞琳 <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I am trying to integrate Flume NG and zookeeper. However, I did not
>>>>> find any configuration about this in Flume NG. Could you please advise how
>>>>> to deal with this?
>>>>>
>>>>> Thanks,
>>>>> RL
>>>>
>>>>
>>>>
>>>
>>
>
+
Denny Ye 2013-02-10, 09:13