Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Flume is not adding .tmp suffix for in-progress files


Copy link to this message
-
Flume is not adding .tmp suffix for in-progress files
Lenin Raj 2013-10-25, 20:43
Hi All,

I am having a simple agent config like below:

agent.sources = Twitter
agent.channels = MemChannel
agent.sinks = FileSink

agent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource
agent.sources.Twitter.channels = MemChannel

agent.sinks.FileSink.channel = MemChannel
agent.sinks.FileSink.type = file_roll
agent.sinks.FileSink.sink.directory = /var/data/
agent.sinks.FileSink.sink.rollInterval = 60

agent.channels.MemChannel.type = memory
agent.channels.MemChannel.capacity = 50000
agent.channels.MemChannel.transactionCapacity = 1000
root@sys:/var/data# ls -ltrh
total 8.0K
-rw-r--r-- 1 root root 4.9K Oct 26 02:00 1382733013173-1
-rw-r--r-- 1 root root    0 Oct 26 02:01 1382733013173-2
root@ sys:/var/data# ls -ltrh
total 12K
-rw-r--r-- 1 root root 4.9K Oct 26 02:00 1382733013173-1
-rw-r--r-- 1 root root 2.1K Oct 26 02:01 1382733013173-2

As you can see, the file "1382733013173-2" is being modified without the
".tmp" prefix.

My Flume version:
Flume 1.4.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git

Does the config file need any modifications to get .tmp suffix? I need this
suffix to exclude this file from my file watcher script.

Thanks,
Lenin