Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Spooling...


Copy link to this message
-
Re: Spooling...
Good catch https://issues.apache.org/jira/browse/FLUME-1705 yes we'll
need to get this into 1.3.

On Tue, Nov 13, 2012 at 4:10 PM, Will McQueen <[EMAIL PROTECTED]> wrote:
> more background info...
>
> SourceType.java says:
>
>   SPOOLDIR("org.apache.flume.conf.source.SpoolDirectorySource");
>
> ...but should be:
>
>   SPOOLDIR("org.apache.flume.source.SpoolDirectorySource");
>
> ...so for now, just specify this in your conf file:
>
> a1.sources.r1.type = org.apache.flume.source.SpoolDirectorySource
>
> ...instead of this:
>
> a1.sources.r1.type = SPOOLDIR
>
> I think we should get this change in for 1.3.0
>
>
> On Tue, Nov 13, 2012 at 2:08 PM, Will McQueen <[EMAIL PROTECTED]> wrote:
>>
>> SourceType class needs to be updated to point to the correct FQCN. For
>> now, just specify the FQCN as the type.
>>
>>
>> On Tue, Nov 13, 2012 at 1:42 PM, Dan Young <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>>
>>> I'm curious where FLUME-1425 stands?  I see it resolved in 1.3, so I
>>> downloaded the branch from git, ran the mvn install, but when I try to
>>> setup/run with an agent.source.x.type = spooldir (per the 1.3
>>> documentation), I'm getting  the following exception. Any advice/insight
>>> would be greatly appreciated....
>>>
>>>
>>>
>>>
>>>
>>> org.apache.flume.FlumeException: Unable to load source type: spooldir,
>>> class: org.apache.flume.conf.source.SpoolDirectorySource
>>> at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:93)
>>> at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:319)
>>> at
>>> org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38)
>>> at
>>> org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>> at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:680)
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.flume.conf.source.SpoolDirectorySource
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:169)
>>> at
>>> org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:91)
>>> ... 14 more
>>>
>>>
>>> Thanx,
>>>
>>> Dano
>>>
>>
>

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB