Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Can we use escape sequences in file_roll sink?


+
Lenin Raj 2013-06-14, 19:04
Copy link to this message
-
Re: Can we use escape sequences in file_roll sink?
Hi Lenin,

Instead of using the file_roll sink, we just use the HDFS sink with a
local path and it works fine. Could that work for you

On 06/15/2013 04:04 AM, Lenin Raj wrote:
> Hello,
>
> I tried to use %Y in my sink.directory in the file_roll sink but it
> resulted in exception:
>
> 15 Jun 2013 00:26:20,596 ERROR
> [SinkRunner-PollingRunner-DefaultSinkProcessor]
> (org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to
> deliver event. Exception follows.
> org.apache.flume.EventDeliveryException: Failed to open file
> /var/emaillenin/card/data/%Y/1371236170080-1 while delivering event
>         at
> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:177)
>         at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>         at
> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.FileNotFoundException:
> /var/emaillenin/card/data/%Y/1371236170080-1 (No such file or directory)
>         at java.io.FileOutputStream.open(Native Method)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:145)
>         at
> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
>         ... 3 more
>
> When I remove %Y from the path and tried again, the files get created
> successfuly. Are there any ways to use these escape sequences in
> file_roll sink?
>
> Thanks,
> Lenin
+
Lenin Raj 2013-06-17, 02:31
+
Hari Shreedharan 2013-06-14, 19:10
+
Lenin Raj 2013-06-14, 19:22
+
Hari Shreedharan 2013-06-14, 19:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB