+
Jeff Bachtel 2013-01-15, 01:57
+
Connor Woodson 2013-01-15, 02:06
+
Jeff Bachtel 2013-01-15, 02:12
+
Connor Woodson 2013-01-15, 02:17
-
Re: Unable to activate flume-ng-cassandra-sinkBrock Noland 2013-01-15, 02:23
Great to hear the plugin directory is being used. Just to confirm, do
you have the following:
/usr/local/flume/plugins.d/flume-ng-cassandra-sink/lib/
correct?
Generally the plugin.d was created so this:
/usr/local/flume/plugins.d/plugin/lib/
would have the plugin jar and this:
/usr/local/flume/plugins.d/plugin/libext/
would have any jars the plugin depended on.
Brock
On Mon, Jan 14, 2013 at 6:17 PM, Connor Woodson <[EMAIL PROTECTED]> wrote:
> The reason I don't recommend using it is that I didn't know it existed :P
>
> That seems like a great place to put it. I've done quite a bit of
> customization, so for me it's easier to just point Flume to the ./target
> directory of my java projects. But the plugins.d seems like a great place
> for third-party libraries.
>
> Glad I could help!
>
> - Connor
>
>
> On Mon, Jan 14, 2013 at 6:12 PM, Jeff Bachtel <[EMAIL PROTECTED]>
> wrote:
>>
>> Awesome, thanks; it was that typo.
>>
>> The plugins.d directory seems to be working, is there a particular reason
>> you don't recommend using? I'm brand-spanking new to Flume, so I'm curious.
>>
>> Thanks again,
>>
>> Jeff
>>
>>
>> On Mon, Jan 14, 2013 at 9:06 PM, Connor Woodson <[EMAIL PROTECTED]>
>> wrote:
>>>
>>> Leave the Cassandra sink jar with its other jars in a folder some where.
>>> In FLUME_HOME/conf, open flume-env.sh and go to the FLUME_CLASSPATH line.
>>> Uncomment it if needed. Append the path to your cassandra sink and its
>>> associated jars, and see if that works.
>>>
>>> There doesn't appear to be a version incompatability; simply, the
>>> Cassandra jar is not found in the classpath. And the reason for that is it
>>> looks like you have a typo in your flume.conf.
>>>
>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>>>
>>> should probably be
>>>
>>> com.btoddb.flume.sinks.cassandra.CassandraSink
>>>
>>> - Connor
>>>
>>>
>>> On Mon, Jan 14, 2013 at 5:57 PM, Jeff Bachtel <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>> I have a build of Flume 1.4.0 on FreeBSD 8, installed into
>>>> /usr/local/flume . I manually created
>>>> /usr/local/flume/plugins.d/flume-ng-cassandra-sink and untarred
>>>> flume-ng-cassandra-sink-1.0.0-SNAPSHOT-dist.tar.gz (from a local pull/build
>>>> of btoddb's sources on github), so I have a lib/ directory that's completely
>>>> populated with both the sink plugin and any possible jarfiles it might rely
>>>> on.
>>>>
>>>> I've likewise tried just copying
>>>> flume-ng-cassandra-sink-1.0.0-SNAPSHOT.jar to flume/lib/ for it to get
>>>> picked up.
>>>>
>>>> When I have a CassandraSink define in flume.conf as such:
>>>> webserver.sinks.cassandraSink.type >>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>>>> webserver.sinks.cassandraSink.channel = memoryChannel
>>>> webserver.sinks.cassandraSink.hosts = localhost
>>>> webserver.sinks.cassandraSink.cluster-name = MyCluster
>>>> webserver.sinks.cassandraSink.keyspace-name = ApacheLogs
>>>> webserver.sinks.cassandraSink.records-coldfam = Requests
>>>>
>>>> and try to start Flume with:
>>>>
>>>> /usr/local/flume/bin/flume-ng agent -c /usr/local/flume/conf/ -f
>>>> /usr/local/flume/conf/flume.conf -n webserver
>>>> -Dflume.root.logger=DEBUG,console
>>>>
>>>>
>>>> I get the following output (and obviously, no success):
>>>> 2013-01-14 19:55:33,594 (conf-file-poller-0) [ERROR -
>>>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:142)]
>>>> Failed to load configuration data. Exception follows.
>>>> org.apache.flume.FlumeException: Unable to load sink type:
>>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink, class:
>>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>>>> at
>>>> org.apache.flume.sink.DefaultSinkFactory.getClass(DefaultSinkFactory.java:69)
>>>> at
>>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:41)
>>>> at
>>>> org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:415)
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
you have the following:
/usr/local/flume/plugins.d/flume-ng-cassandra-sink/lib/
correct?
Generally the plugin.d was created so this:
/usr/local/flume/plugins.d/plugin/lib/
would have the plugin jar and this:
/usr/local/flume/plugins.d/plugin/libext/
would have any jars the plugin depended on.
Brock
On Mon, Jan 14, 2013 at 6:17 PM, Connor Woodson <[EMAIL PROTECTED]> wrote:
> The reason I don't recommend using it is that I didn't know it existed :P
>
> That seems like a great place to put it. I've done quite a bit of
> customization, so for me it's easier to just point Flume to the ./target
> directory of my java projects. But the plugins.d seems like a great place
> for third-party libraries.
>
> Glad I could help!
>
> - Connor
>
>
> On Mon, Jan 14, 2013 at 6:12 PM, Jeff Bachtel <[EMAIL PROTECTED]>
> wrote:
>>
>> Awesome, thanks; it was that typo.
>>
>> The plugins.d directory seems to be working, is there a particular reason
>> you don't recommend using? I'm brand-spanking new to Flume, so I'm curious.
>>
>> Thanks again,
>>
>> Jeff
>>
>>
>> On Mon, Jan 14, 2013 at 9:06 PM, Connor Woodson <[EMAIL PROTECTED]>
>> wrote:
>>>
>>> Leave the Cassandra sink jar with its other jars in a folder some where.
>>> In FLUME_HOME/conf, open flume-env.sh and go to the FLUME_CLASSPATH line.
>>> Uncomment it if needed. Append the path to your cassandra sink and its
>>> associated jars, and see if that works.
>>>
>>> There doesn't appear to be a version incompatability; simply, the
>>> Cassandra jar is not found in the classpath. And the reason for that is it
>>> looks like you have a typo in your flume.conf.
>>>
>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>>>
>>> should probably be
>>>
>>> com.btoddb.flume.sinks.cassandra.CassandraSink
>>>
>>> - Connor
>>>
>>>
>>> On Mon, Jan 14, 2013 at 5:57 PM, Jeff Bachtel <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>> I have a build of Flume 1.4.0 on FreeBSD 8, installed into
>>>> /usr/local/flume . I manually created
>>>> /usr/local/flume/plugins.d/flume-ng-cassandra-sink and untarred
>>>> flume-ng-cassandra-sink-1.0.0-SNAPSHOT-dist.tar.gz (from a local pull/build
>>>> of btoddb's sources on github), so I have a lib/ directory that's completely
>>>> populated with both the sink plugin and any possible jarfiles it might rely
>>>> on.
>>>>
>>>> I've likewise tried just copying
>>>> flume-ng-cassandra-sink-1.0.0-SNAPSHOT.jar to flume/lib/ for it to get
>>>> picked up.
>>>>
>>>> When I have a CassandraSink define in flume.conf as such:
>>>> webserver.sinks.cassandraSink.type >>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>>>> webserver.sinks.cassandraSink.channel = memoryChannel
>>>> webserver.sinks.cassandraSink.hosts = localhost
>>>> webserver.sinks.cassandraSink.cluster-name = MyCluster
>>>> webserver.sinks.cassandraSink.keyspace-name = ApacheLogs
>>>> webserver.sinks.cassandraSink.records-coldfam = Requests
>>>>
>>>> and try to start Flume with:
>>>>
>>>> /usr/local/flume/bin/flume-ng agent -c /usr/local/flume/conf/ -f
>>>> /usr/local/flume/conf/flume.conf -n webserver
>>>> -Dflume.root.logger=DEBUG,console
>>>>
>>>>
>>>> I get the following output (and obviously, no success):
>>>> 2013-01-14 19:55:33,594 (conf-file-poller-0) [ERROR -
>>>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:142)]
>>>> Failed to load configuration data. Exception follows.
>>>> org.apache.flume.FlumeException: Unable to load sink type:
>>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink, class:
>>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>>>> at
>>>> org.apache.flume.sink.DefaultSinkFactory.getClass(DefaultSinkFactory.java:69)
>>>> at
>>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:41)
>>>> at
>>>> org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:415)
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Jeff Bachtel 2013-01-15, 17:12
+
Brock Noland 2013-01-15, 17:21
