Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Spooling fileSuffix attribute ignored


Copy link to this message
-
Spooling fileSuffix attribute ignored
Dear all,
I am using the Spooling attribute "fileSuffix" to ensure that only when a file has been completely upload into the Spooling dir,  is consumed by the source. However, files with no suffix are also appended to the channel and then processed by the sink.
My configuration:

tier1.sources  = s1
tier1.channels = c1
tier1.sinks    = s1

# For each source, channel, and sink, set
# standard properties.
tier1.sources.s1.type     = spooldir
tier1.sources.s1.spoolDir = /home/user/flume/data
tier1.sources.s1.deletePolicy = immediate
tier1.sources.s1.batchSize = 1000
tier1.sources.s1.bufferMaxLines = 3000
tier1.sources.s1.fileHeader = true
tier1.sources.s1.fileSuffix=.COMPLETED

Is that ok?

Thanks.

________________________________

Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra pol?tica de env?o y recepci?n de correo electr?nico en el enlace situado m?s abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx