Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Flume idea


Copy link to this message
-
Re: Flume idea
Could you elaborate on "using memcached between all the Tier 1 flume
servers."
It is not clear where memcached is fitting in the data flow pipeline.
-roshan
On Thu, Sep 26, 2013 at 2:08 AM, Rami Mankevich <[EMAIL PROTECTED]>wrote:

> Hey Guys.
> I m working iwth flume for couple of months and faces several issues:
> My flow is simple:
> Im using REST WS which has internal AVRO client.
> So the flow is simple :
> HTTP webservce -> AVRO CLIENT ------------> Fume with AVRO source
> The problem starts when i started using File Channel Since now all the HTTP
> requests to the REST WS are blocked on file system .
> With memory channel evrything was fast and perfect,
> So i started thinking about solution and the idea was of using memcached
> between all the Tier 1 flume servers.
> This way i will not be blocked on file system from one side and from other
> side i will still be hightly avalible.
> Does it make sence?
>
> In general i implemented teh solution....
> Just wanted to know you opinion.
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.