Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Flume - Custom HDFS Sink does not write to hdfs unless killed


+
Himanshu Patidar 2014-03-03, 09:12
Copy link to this message
-
Re: Flume - Custom HDFS Sink does not write to hdfs unless killed
Not sure if this is the issue but I've found it's easy to have custom flume
components like interceptors, sinks, sources and serializers swallow
exceptions.  You won't see any errors in standard out and the flume agent
will look like it is working but it's not doing anything.  When you kill
the agent it will then close your files in HDFS and any data that was in
the channel before the error will get written out.  Do the avro fies have
all the data you expect them to have after you kill the flume agent?

Best Regards,

Ed
On Mon, Mar 3, 2014 at 4:11 AM, Himanshu Patidar <
[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB