Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Flume idea


Hey Guys.
I m working iwth flume for couple of months and faces several issues:
My flow is simple:
Im using REST WS which has internal AVRO client.
So the flow is simple :
HTTP webservce -> AVRO CLIENT ------------> Fume with AVRO source
The problem starts when i started using File Channel Since now all the HTTP
requests to the REST WS are blocked on file system .
With memory channel evrything was fast and perfect,
So i started thinking about solution and the idea was of using memcached
between all the Tier 1 flume servers.
This way i will not be blocked on file system from one side and from other
side i will still be hightly avalible.
Does it make sence?

In general i implemented teh solution....
Just wanted to know you opinion.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB