Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Unable to activate flume-ng-cassandra-sink


+
Jeff Bachtel 2013-01-15, 01:57
+
Connor Woodson 2013-01-15, 02:06
+
Jeff Bachtel 2013-01-15, 02:12
Copy link to this message
-
Re: Unable to activate flume-ng-cassandra-sink
The reason I don't recommend using it is that I didn't know it existed :P

That seems like a great place to put it. I've done quite a bit of
customization, so for me it's easier to just point Flume to the ./target
directory of my java projects. But the plugins.d seems like a great place
for third-party libraries.

Glad I could help!

- Connor
On Mon, Jan 14, 2013 at 6:12 PM, Jeff Bachtel <[EMAIL PROTECTED]>wrote:

> Awesome, thanks; it was that typo.
>
> The plugins.d directory seems to be working, is there a particular reason
> you don't recommend using? I'm brand-spanking new to Flume, so I'm curious.
>
> Thanks again,
>
> Jeff
>
>
> On Mon, Jan 14, 2013 at 9:06 PM, Connor Woodson <[EMAIL PROTECTED]>wrote:
>
>> Leave the Cassandra sink jar with its other jars in a folder some where.
>> In FLUME_HOME/conf, open flume-env.sh and go to the FLUME_CLASSPATH line.
>> Uncomment it if needed. Append the path to your cassandra sink and its
>> associated jars, and see if that works.
>>
>> There doesn't appear to be a version incompatability; simply, the
>> Cassandra jar is not found in the classpath. And the reason for that is it
>> looks like you have a typo in your flume.conf.
>>
>> com.*t*btoddb.flume.sinks.cassandra.CassandraSink
>>
>> should probably be
>>
>> com.btoddb.flume.sinks.cassandra.CassandraSink
>>
>> - Connor
>>
>>
>> On Mon, Jan 14, 2013 at 5:57 PM, Jeff Bachtel <[EMAIL PROTECTED]>wrote:
>>
>>> I have a build of Flume 1.4.0 on FreeBSD 8, installed into
>>> /usr/local/flume . I manually created
>>> /usr/local/flume/plugins.d/flume-ng-cassandra-sink and
>>> untarred flume-ng-cassandra-sink-1.0.0-SNAPSHOT-dist.tar.gz (from a local
>>> pull/build of btoddb's sources on github), so I have a lib/ directory
>>> that's completely populated with both the sink plugin and any possible
>>> jarfiles it might rely on.
>>>
>>> I've likewise tried just
>>> copying flume-ng-cassandra-sink-1.0.0-SNAPSHOT.jar to flume/lib/ for it to
>>> get picked up.
>>>
>>> When I have a CassandraSink define in flume.conf as such:
>>> webserver.sinks.cassandraSink.type >>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>>> webserver.sinks.cassandraSink.channel = memoryChannel
>>> webserver.sinks.cassandraSink.hosts = localhost
>>> webserver.sinks.cassandraSink.cluster-name = MyCluster
>>> webserver.sinks.cassandraSink.keyspace-name = ApacheLogs
>>> webserver.sinks.cassandraSink.records-coldfam = Requests
>>>
>>> and try to start Flume with:
>>>
>>> /usr/local/flume/bin/flume-ng agent -c /usr/local/flume/conf/ -f
>>> /usr/local/flume/conf/flume.conf -n webserver
>>> -Dflume.root.logger=DEBUG,console
>>>
>>>
>>> I get the following output (and obviously, no success):
>>> 2013-01-14 19:55:33,594 (conf-file-poller-0) [ERROR -
>>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:142)]
>>> Failed to load configuration data. Exception follows.
>>> org.apache.flume.FlumeException: Unable to load sink type:
>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink, class:
>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>>>         at
>>> org.apache.flume.sink.DefaultSinkFactory.getClass(DefaultSinkFactory.java:69)
>>>         at
>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:41)
>>>         at
>>> org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:415)
>>>          at
>>> org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:103)
>>>         at
>>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
>>>         at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>>>         at
>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
+
Brock Noland 2013-01-15, 02:23
+
Jeff Bachtel 2013-01-15, 17:12
+
Brock Noland 2013-01-15, 17:21