Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Flume idea


Hey Guys.
I m working iwth flume for couple of months and faces several issues:
My flow is simple:
Im using REST WS which has internal AVRO client.
So the flow is simple :
HTTP webservce -> AVRO CLIENT ------------> Fume with AVRO source
The problem starts when i started using File Channel Since now all the HTTP
requests to the REST WS are blocked on file system .
With memory channel evrything was fast and perfect,
So i started thinking about solution and the idea was of using memcached
between all the Tier 1 flume servers.
This way i will not be blocked on file system from one side and from other
side i will still be hightly avalible.
Does it make sence?

In general i implemented teh solution....
Just wanted to know you opinion.