Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> doubt on hdfs sink

Copy link to this message
doubt on hdfs sink

I have a working setup of flume which writes into hdfs.
I am using flum 1.2.0

I have following settings.

agent1.sinks.HDFS.hdfs.file.Type = SequenceFile
agent1.sinks.HDFS.hdfs.writeFormat = Text
agent1.sinks.hdfs.rollInterval = 86400
agent1.channels.MemoryChannel-2.type = memory

My doubt is, why does flume creates one hdfs file for one event. I
want it to write a single hdfs file per day for the log.
Can someone please help me find out what I have done wrong?


Nitin Pawar
Nitin Pawar 2012-09-18, 09:02