Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Not able to set date %d%m%Y in Sink Directory name


Copy link to this message
-
Not able to set date %d%m%Y in Sink Directory name
Hi ,

I am trying to store my logs in folders named with date for my file_roll
and hdfssink. For some reason when I pass %d%m%Y in the sink directory it
is not working . Any thoughts .

My Flume source is a simple HTTP Handler extended from HTTPSourceHandler

tier1.sinks.filesink1.type = file_roll
tier1.sinks.filesink1.channel = c2
tier1.sinks.filesink1.batchSize = 100000
tier1.sinks.filesink1.rollInterval = 0
#tier1.sinks.filesink1.rollInterval = 86400
tier1.sinks.filesink1.sink.directory = /projects/test/logs/%d%m%Y
13/10/23 16:20:36 ERROR flume.SinkRunner: Unable to deliver event.
Exception follows.
org.apache.flume.EventDeliveryException: Failed to open file
/projects/test/logs/%d%m%Y/1382541251307-1 while delivering event
at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:177)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.FileNotFoundException:
/projects/ETHAN_FLUME/logs/%d%m%Y/1382541251307-1 (No such file or
directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
Thanks,
Deepak Subhramanian
+
David Sinclair 2013-10-23, 17:40
+
Deepak Subhramanian 2013-10-24, 09:16
+
David Sinclair 2013-10-24, 13:47
+
Deepak Subhramanian 2013-10-24, 14:55