Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Not able to set date %d%m%Y in Sink Directory name


Copy link to this message
-
Not able to set date %d%m%Y in Sink Directory name
Hi ,

I am trying to store my logs in folders named with date for my file_roll
and hdfssink. For some reason when I pass %d%m%Y in the sink directory it
is not working . Any thoughts .

My Flume source is a simple HTTP Handler extended from HTTPSourceHandler

tier1.sinks.filesink1.type = file_roll
tier1.sinks.filesink1.channel = c2
tier1.sinks.filesink1.batchSize = 100000
tier1.sinks.filesink1.rollInterval = 0
#tier1.sinks.filesink1.rollInterval = 86400
tier1.sinks.filesink1.sink.directory = /projects/test/logs/%d%m%Y
13/10/23 16:20:36 ERROR flume.SinkRunner: Unable to deliver event.
Exception follows.
org.apache.flume.EventDeliveryException: Failed to open file
/projects/test/logs/%d%m%Y/1382541251307-1 while delivering event
at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:177)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.FileNotFoundException:
/projects/ETHAN_FLUME/logs/%d%m%Y/1382541251307-1 (No such file or
directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
Thanks,
Deepak Subhramanian
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB