Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Can we use escape sequences in file_roll sink?


Copy link to this message
-
Re: Can we use escape sequences in file_roll sink?
Lenin Raj 2013-06-17, 02:31
Hi Juhani,

I have not tried with that. But that is not the recommended way right? We
might have additional properties for file_roll sink in the future, maybe.
Thanks,
Lenin
On Mon, Jun 17, 2013 at 7:45 AM, Juhani Connolly <
[EMAIL PROTECTED]> wrote:

> Hi Lenin,
>
> Instead of using the file_roll sink, we just use the HDFS sink with a
> local path and it works fine. Could that work for you
>
>
> On 06/15/2013 04:04 AM, Lenin Raj wrote:
>
>> Hello,
>>
>> I tried to use %Y in my sink.directory in the file_roll sink but it
>> resulted in exception:
>>
>> 15 Jun 2013 00:26:20,596 ERROR [SinkRunner-PollingRunner-**DefaultSinkProcessor]
>> (org.apache.flume.SinkRunner$**PollingRunner.run:160)  - Unable to
>> deliver event. Exception follows.
>> org.apache.flume.**EventDeliveryException: Failed to open file
>> /var/emaillenin/card/data/%Y/**1371236170080-1 while delivering event
>>         at org.apache.flume.sink.**RollingFileSink.process(**
>> RollingFileSink.java:177)
>>         at org.apache.flume.sink.**DefaultSinkProcessor.process(**
>> DefaultSinkProcessor.java:68)
>>         at org.apache.flume.SinkRunner$**PollingRunner.run(SinkRunner.**
>> java:147)
>>         at java.lang.Thread.run(Thread.**java:662)
>> Caused by: java.io.FileNotFoundException: /var/emaillenin/card/data/%Y/**1371236170080-1
>> (No such file or directory)
>>         at java.io.FileOutputStream.open(**Native Method)
>>         at java.io.FileOutputStream.<**init>(FileOutputStream.java:**194)
>>         at java.io.FileOutputStream.<**init>(FileOutputStream.java:**145)
>>         at org.apache.flume.sink.**RollingFileSink.process(**
>> RollingFileSink.java:169)
>>         ... 3 more
>>
>> When I remove %Y from the path and tried again, the files get created
>> successfuly. Are there any ways to use these escape sequences in file_roll
>> sink?
>>
>> Thanks,
>> Lenin
>>
>
>