Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Can we use escape sequences in file_roll sink?


Copy link to this message
-
Re: Can we use escape sequences in file_roll sink?
The main implementation is in the BucketPath class. It should not be too difficult to create the path based on that.
Cheers,
Hari
On Friday, June 14, 2013 at 12:22 PM, Lenin Raj wrote:

> Hi Hari,
>
> Here is the ticket - https://issues.apache.org/jira/browse/FLUME-1902
>
> I will see how it is done in HDFS sink.
>
>
> Thanks,
> Lenin
>
>
> On Sat, Jun 15, 2013 at 12:40 AM, Hari Shreedharan <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > Currently escaping is  not supported by File Roll Sink, but I think there is a jira to enable it - perhaps you want to submit a patch?
> >
> >
> > Cheers,
> > Hari
> >
> >
> > On Friday, June 14, 2013 at 12:04 PM, Lenin Raj wrote:
> >
> > > Hello,
> > >
> > > I tried to use %Y in my sink.directory in the file_roll sink but it resulted in exception:
> > >
> > > 15 Jun 2013 00:26:20,596 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to deliver event. Exception follows.
> > > org.apache.flume.EventDeliveryException: Failed to open file /var/emaillenin/card/data/%Y/1371236170080-1 while delivering event
> > >         at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:177)
> > >         at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> > >         at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> > >         at java.lang.Thread.run(Thread.java:662)
> > > Caused by: java.io.FileNotFoundException: /var/emaillenin/card/data/%Y/1371236170080-1 (No such file or directory)
> > >         at java.io.FileOutputStream.open(Native Method)
> > >         at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
> > >         at java.io.FileOutputStream.<init>(FileOutputStream.java:145)
> > >         at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:169)
> > >         ... 3 more
> > >
> > > When I remove %Y from the path and tried again, the files get created successfuly. Are there any ways to use these escape sequences in file_roll sink?
> > >
> > > Thanks,
> > > Lenin
> >
>