Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - SpoolDir sub-directories


+
Tzur Turkenitz 2013-01-31, 16:56
Copy link to this message
-
Re: SpoolDir sub-directories
Tzur Turkenitz 2013-02-04, 16:01
Hi,
I couldn't find any information about this in the Wiki / User guide.
Are sub directories for the Flume SpoolDir supported in 1.3x?
Thanks!
On Thu, Jan 31, 2013 at 11:56 AM, Tzur Turkenitz <[EMAIL PROTECTED]> wrote:

> Hi All,
>
> I am experimenting with Flumes` 1.3 new SpoolDir adapter.
> I was wondering is there an option to tell the adapter to walk down
> directories and search for files.
>
> Currently, if I have a directory within the source location I get java
> exceptions.
> 13/01/30 17:41:30 WARN avro.SpoolingFileLineReader: Could not find file:
> /home/fl/source/dir
> java.io.FileNotFoundException: /home/fl/source/dir (Is a directory)
>  at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.<init>(FileInputStream.java:120)
>  at java.io.FileReader.<init>(FileReader.java:55)
> at
> org.apache.flume.client.avro.SpoolingFileLineReader.getNextFile(SpoolingFileLineReader.java:322)
>  at
> org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:172)
> at
> org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:135)
>  at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:662)
> Thanks!
>
>
>
> --
> Regards,
> Tzur Turkenitz
> Vision.BI
> http://www.vision.bi/
>
> "*Facts are stubborn things, but statistics are more pliable*"
> -Mark Twain
>

--
Regards,
Tzur Turkenitz
Vision.BI
http://www.vision.bi/

"*Facts are stubborn things, but statistics are more pliable*"
-Mark Twain
+
Brock Noland 2013-02-04, 16:13
+
Tzur Turkenitz 2013-02-04, 16:41