Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume is not adding .tmp suffix for in-progress files


Copy link to this message
-
Flume is not adding .tmp suffix for in-progress files
Hi All,

I am having a simple agent config like below:

agent.sources = Twitter
agent.channels = MemChannel
agent.sinks = FileSink

agent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource
agent.sources.Twitter.channels = MemChannel

agent.sinks.FileSink.channel = MemChannel
agent.sinks.FileSink.type = file_roll
agent.sinks.FileSink.sink.directory = /var/data/
agent.sinks.FileSink.sink.rollInterval = 60

agent.channels.MemChannel.type = memory
agent.channels.MemChannel.capacity = 50000
agent.channels.MemChannel.transactionCapacity = 1000
root@sys:/var/data# ls -ltrh
total 8.0K
-rw-r--r-- 1 root root 4.9K Oct 26 02:00 1382733013173-1
-rw-r--r-- 1 root root    0 Oct 26 02:01 1382733013173-2
root@ sys:/var/data# ls -ltrh
total 12K
-rw-r--r-- 1 root root 4.9K Oct 26 02:00 1382733013173-1
-rw-r--r-- 1 root root 2.1K Oct 26 02:01 1382733013173-2

As you can see, the file "1382733013173-2" is being modified without the
".tmp" prefix.

My Flume version:
Flume 1.4.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git

Does the config file need any modifications to get .tmp suffix? I need this
suffix to exclude this file from my file watcher script.

Thanks,
Lenin
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB