Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Spooling...


Copy link to this message
-
Re: Spooling...
Brock Noland 2012-11-13, 22:12
Good catch https://issues.apache.org/jira/browse/FLUME-1705 yes we'll
need to get this into 1.3.

On Tue, Nov 13, 2012 at 4:10 PM, Will McQueen <[EMAIL PROTECTED]> wrote:
> more background info...
>
> SourceType.java says:
>
>   SPOOLDIR("org.apache.flume.conf.source.SpoolDirectorySource");
>
> ...but should be:
>
>   SPOOLDIR("org.apache.flume.source.SpoolDirectorySource");
>
> ...so for now, just specify this in your conf file:
>
> a1.sources.r1.type = org.apache.flume.source.SpoolDirectorySource
>
> ...instead of this:
>
> a1.sources.r1.type = SPOOLDIR
>
> I think we should get this change in for 1.3.0
>
>
> On Tue, Nov 13, 2012 at 2:08 PM, Will McQueen <[EMAIL PROTECTED]> wrote:
>>
>> SourceType class needs to be updated to point to the correct FQCN. For
>> now, just specify the FQCN as the type.
>>
>>
>> On Tue, Nov 13, 2012 at 1:42 PM, Dan Young <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>>
>>> I'm curious where FLUME-1425 stands?  I see it resolved in 1.3, so I
>>> downloaded the branch from git, ran the mvn install, but when I try to
>>> setup/run with an agent.source.x.type = spooldir (per the 1.3
>>> documentation), I'm getting  the following exception. Any advice/insight
>>> would be greatly appreciated....
>>>
>>>
>>>
>>>
>>>
>>> org.apache.flume.FlumeException: Unable to load source type: spooldir,
>>> class: org.apache.flume.conf.source.SpoolDirectorySource
>>> at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
>>> at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
>>> at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>> at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.flume.conf.source.SpoolDirectorySource
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:169)
>>> at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
>>> ... 14 more
>>>
>>>
>>> Thanx,
>>>
>>> Dano
>>>
>>
>

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/