I'm curious where FLUME-1425 stands?  I see it resolved in 1.3, so I
downloaded the branch from git, ran the mvn install, but when I try to
setup/run with an agent.source.x.type = spooldir (per the 1.3
documentation), I'm getting  the following exception. Any advice/insight
would be greatly appreciated....

org.apache.flume.FlumeException: Unable to load source type: spooldir,
class: org.apache.flume.conf.source.SpoolDirectorySource
at
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
at
org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
at
org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: java.lang.ClassNotFoundException:
org.apache.flume.conf.source.SpoolDirectorySource
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at
org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
... 14 more
Thanx,

Dano

SourceType class needs to be updated to point to the correct FQCN. For now,
just specify the FQCN as the type.

On Tue, Nov 13, 2012 at 1:42 PM, Dan Young <[EMAIL PROTECTED]> wrote:

>
>
> I'm curious where FLUME-1425 stands?  I see it resolved in 1.3, so I
> downloaded the branch from git, ran the mvn install, but when I try to
> setup/run with an agent.source.x.type = spooldir (per the 1.3
> documentation), I'm getting  the following exception. Any advice/insight
> would be greatly appreciated....
>
>
>
>
>
> org.apache.flume.FlumeException: Unable to load source type: spooldir,
> class: org.apache.flume.conf.source.SpoolDirectorySource
>  at
> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
> at
> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
>  at
> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>  at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
> at
> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>  at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>  at java.lang.Thread.run(Thread.java:680)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flume.conf.source.SpoolDirectorySource
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>  at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
>  at
> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
> ... 14 more
>
>
> Thanx,
>
> Dano
>
>

more background info...

SourceType.java says:

  SPOOLDIR("org.apache.flume.conf.source.SpoolDirectorySource");

...but should be:

  SPOOLDIR("org.apache.flume.source.SpoolDirectorySource");

...so for now, just specify this in your conf file:

a1.sources.r1.*type* = org.apache.flume.source.SpoolDirectorySource

...instead of this:

a1.sources.r1.*type* = SPOOLDIR

I think we should get this change in for 1.3.0

On Tue, Nov 13, 2012 at 2:08 PM, Will McQueen <[EMAIL PROTECTED]> wrote:

> SourceType class needs to be updated to point to the correct FQCN. For
> now, just specify the FQCN as the type.
>
>
> On Tue, Nov 13, 2012 at 1:42 PM, Dan Young <[EMAIL PROTECTED]> wrote:
>
>>
>>
>> I'm curious where FLUME-1425 stands?  I see it resolved in 1.3, so I
>> downloaded the branch from git, ran the mvn install, but when I try to
>> setup/run with an agent.source.x.type = spooldir (per the 1.3
>> documentation), I'm getting  the following exception. Any advice/insight
>> would be greatly appreciated....
>>
>>
>>
>>
>>
>> org.apache.flume.FlumeException: Unable to load source type: spooldir,
>> class: org.apache.flume.conf.source.SpoolDirectorySource
>>  at
>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
>> at
>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
>>  at
>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
>> at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>>  at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>> at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>>  at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>  at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>>  at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>  at java.lang.Thread.run(Thread.java:680)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flume.conf.source.SpoolDirectorySource
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>  at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>>  at
>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
>> ... 14 more
>>
>>
>> Thanx,
>>
>> Dano
>>
>>
>

Good catch https://issues.apache.org/jira/browse/FLUME-1705 yes we'll
need to get this into 1.3.

On Tue, Nov 13, 2012 at 4:10 PM, Will McQueen <[EMAIL PROTECTED]> wrote:
> more background info...
>
> SourceType.java says:
>
>   SPOOLDIR("org.apache.flume.conf.source.SpoolDirectorySource");
>
> ...but should be:
>
>   SPOOLDIR("org.apache.flume.source.SpoolDirectorySource");
>
> ...so for now, just specify this in your conf file:
>
> a1.sources.r1.type = org.apache.flume.source.SpoolDirectorySource
>
> ...instead of this:
>
> a1.sources.r1.type = SPOOLDIR
>
> I think we should get this change in for 1.3.0
>
>
> On Tue, Nov 13, 2012 at 2:08 PM, Will McQueen <[EMAIL PROTECTED]> wrote:
>>
>> SourceType class needs to be updated to point to the correct FQCN. For
>> now, just specify the FQCN as the type.
>>
>>
>> On Tue, Nov 13, 2012 at 1:42 PM, Dan Young <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>>
>>> I'm curious where FLUME-1425 stands?  I see it resolved in 1.3, so I
>>> downloaded the branch from git, ran the mvn install, but when I try to
>>> setup/run with an agent.source.x.type = spooldir (per the 1.3
>>> documentation), I'm getting  the following exception. Any advice/insight
>>> would be greatly appreciated....
>>>
>>>
>>>
>>>
>>>
>>> org.apache.flume.FlumeException: Unable to load source type: spooldir,
>>> class: org.apache.flume.conf.source.SpoolDirectorySource
>>> at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
>>> at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
>>> at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>> at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.flume.conf.source.SpoolDirectorySource
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:169)
>>> at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
>>> ... 14 more
>>>
>>>
>>> Thanx,
>>>
>>> Dano
>>>
>>
>

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Great , thank you. I'll try that.

Regards

Dano
On Nov 13, 2012 3:09 PM, "Will McQueen" <[EMAIL PROTECTED]> wrote:

> SourceType class needs to be updated to point to the correct FQCN. For
> now, just specify the FQCN as the type.
>
> On Tue, Nov 13, 2012 at 1:42 PM, Dan Young <[EMAIL PROTECTED]> wrote:
>
>>
>>
>> I'm curious where FLUME-1425 stands?  I see it resolved in 1.3, so I
>> downloaded the branch from git, ran the mvn install, but when I try to
>> setup/run with an agent.source.x.type = spooldir (per the 1.3
>> documentation), I'm getting  the following exception. Any advice/insight
>> would be greatly appreciated....
>>
>>
>>
>>
>>
>> org.apache.flume.FlumeException: Unable to load source type: spooldir,
>> class: org.apache.flume.conf.source.SpoolDirectorySource
>>  at
>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
>> at
>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
>>  at
>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
>> at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>>  at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>> at
>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>>  at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>  at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>>  at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>  at java.lang.Thread.run(Thread.java:680)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.flume.conf.source.SpoolDirectorySource
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>  at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>>  at
>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
>> ... 14 more
>>
>>
>> Thanx,
>>
>> Dano
>>
>>
>

I got it working....pounding on it now :)  Thanx guys....yea, I saw that
typo conf....
On Tue, Nov 13, 2012 at 3:32 PM, Dan Young <[EMAIL PROTECTED]> wrote:

> Great , thank you. I'll try that.
>
> Regards
>
> Dano
> On Nov 13, 2012 3:09 PM, "Will McQueen" <[EMAIL PROTECTED]> wrote:
>
>> SourceType class needs to be updated to point to the correct FQCN. For
>> now, just specify the FQCN as the type.
>>
>> On Tue, Nov 13, 2012 at 1:42 PM, Dan Young <[EMAIL PROTECTED]> wrote:
>>
>>>
>>>
>>> I'm curious where FLUME-1425 stands?  I see it resolved in 1.3, so I
>>> downloaded the branch from git, ran the mvn install, but when I try to
>>> setup/run with an agent.source.x.type = spooldir (per the 1.3
>>> documentation), I'm getting  the following exception. Any advice/insight
>>> would be greatly appreciated....
>>>
>>>
>>>
>>>
>>>
>>> org.apache.flume.FlumeException: Unable to load source type: spooldir,
>>> class: org.apache.flume.conf.source.SpoolDirectorySource
>>>  at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
>>> at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
>>>  at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>>>  at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>>>  at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>> at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>  at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>>>  at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>  at java.lang.Thread.run(Thread.java:680)
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.flume.conf.source.SpoolDirectorySource
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>  at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:169)
>>>  at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
>>> ... 14 more
>>>
>>>
>>> Thanx,
>>>
>>> Dano
>>>
>>>
>>