Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Flume NG and zookeeper


+
吳瑞琳 2013-02-08, 02:36
+
Roshan Naik 2013-02-08, 02:41
+
Brock Noland 2013-02-08, 02:40
+
Mike Percy 2013-02-08, 02:41
+
吳瑞琳 2013-02-08, 03:07
+
Mike Percy 2013-02-08, 03:17
+
吳瑞琳 2013-02-08, 03:50
+
Tech Mail 2013-02-08, 14:44
Copy link to this message
-
Re: Flume NG and zookeeper
you could put flume behind a  VIP and not expose the actual flume nodes
names to clients.
-roshan
On Fri, Feb 8, 2013 at 6:44 AM, Tech Mail <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am looking for similar functionality, but we loose the flexibility.
>
> Cannot add a new agent dynamically to the list of collectors.
> Cannot move the agents from one host to other host or from one datacenter
> to other.
> Entire list of agents has to be provided to all end user applications.
>
> How to dynamically get the host/port of the agent from the client
> application?
>    LDAP server server.
>     Any other ways ?
>
> Yogi
>
> On Feb 7, 2013, at 7:50 PM, 吳瑞琳 <[EMAIL PROTECTED]> wrote:
>
> Mike,
>
> Thanks a lot. It fits our need. I'll try your advice.
>
> Regards,
> RL
>
> 2013/2/8 Mike Percy <[EMAIL PROTECTED]>
>
>> Interesting use case. Have you considered using the existing
>> polling-based load balancing RPC client functionality?
>>
>> http://flume.apache.org/FlumeDeveloperGuide.html#loadbalancing-rpc-client
>>
>> Regards,
>> Mike
>>
>>
>> On Thu, Feb 7, 2013 at 7:07 PM, 吳瑞琳 <[EMAIL PROTECTED]> wrote:
>>
>>> Thank you all.
>>>
>>> I am trying configure multiple hosts running Flume to collect data from
>>> clients. I want to use Zookeeper to track status of Flume hosts so that my
>>> clients can get the list of available Flume host and send data to one of
>>> them. I would like to make Flume hosts to register an ephemeral node under
>>> some Zookeeper node when it is up.
>>>
>>> Regards,
>>> RL
>>>
>>>
>>> 2013/2/8 Mike Percy <[EMAIL PROTECTED]>
>>>
>>>> Integrate in what way?
>>>>
>>>>
>>>> On Thu, Feb 7, 2013 at 6:36 PM, 吳瑞琳 <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I am trying to integrate Flume NG and zookeeper. However, I did not
>>>>> find any configuration about this in Flume NG. Could you please advise how
>>>>> to deal with this?
>>>>>
>>>>> Thanks,
>>>>> RL
>>>>
>>>>
>>>>
>>>
>>
>
+
Denny Ye 2013-02-10, 09:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB