Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Unable to activate flume-ng-cassandra-sink


+
Jeff Bachtel 2013-01-15, 01:57
+
Connor Woodson 2013-01-15, 02:06
+
Jeff Bachtel 2013-01-15, 02:12
+
Connor Woodson 2013-01-15, 02:17
+
Brock Noland 2013-01-15, 02:23
+
Jeff Bachtel 2013-01-15, 17:12
Copy link to this message
-
Re: Unable to activate flume-ng-cassandra-sink
Hi,

We haven't documented it yet as it's not in any release. I've been
meaning to do that! :)

I just created https://issues.apache.org/jira/browse/FLUME-1845 for this.

Brock

On Tue, Jan 15, 2013 at 9:12 AM, Jeff Bachtel <[EMAIL PROTECTED]> wrote:
> No, I hadn't separated the jarfiles out in that manner (I didn't realize the
> order they appeared in the CLASSPATH mattered). I'll certainly go change
> that now.
>
> Is there documentation on the plugins.d directory anywhere? I discovered it
> in the code by accident, and it met my needs while using this custom sink.
>
> Jeff
>
>
> On Mon, Jan 14, 2013 at 9:23 PM, Brock Noland <[EMAIL PROTECTED]> wrote:
>>
>> Great to hear the plugin directory is being used. Just to confirm, do
>> you have the following:
>>
>> /usr/local/flume/plugins.d/flume-ng-cassandra-sink/lib/
>>
>> correct?
>>
>> Generally the plugin.d was created so this:
>>
>> /usr/local/flume/plugins.d/plugin/lib/
>>
>> would have the plugin jar and this:
>>
>> /usr/local/flume/plugins.d/plugin/libext/
>>
>> would have any jars the plugin depended on.
>>
>> Brock
>>
>> On Mon, Jan 14, 2013 at 6:17 PM, Connor Woodson <[EMAIL PROTECTED]>
>> wrote:
>> > The reason I don't recommend using it is that I didn't know it existed
>> > :P
>> >
>> > That seems like a great place to put it. I've done quite a bit of
>> > customization, so for me it's easier to just point Flume to the ./target
>> > directory of my java projects. But the plugins.d seems like a great
>> > place
>> > for third-party libraries.
>> >
>> > Glad I could help!
>> >
>> > - Connor
>> >
>> >
>> > On Mon, Jan 14, 2013 at 6:12 PM, Jeff Bachtel <[EMAIL PROTECTED]>
>> > wrote:
>> >>
>> >> Awesome, thanks; it was that typo.
>> >>
>> >> The plugins.d directory seems to be working, is there a particular
>> >> reason
>> >> you don't recommend using? I'm brand-spanking new to Flume, so I'm
>> >> curious.
>> >>
>> >> Thanks again,
>> >>
>> >> Jeff
>> >>
>> >>
>> >> On Mon, Jan 14, 2013 at 9:06 PM, Connor Woodson
>> >> <[EMAIL PROTECTED]>
>> >> wrote:
>> >>>
>> >>> Leave the Cassandra sink jar with its other jars in a folder some
>> >>> where.
>> >>> In FLUME_HOME/conf, open flume-env.sh and go to the FLUME_CLASSPATH
>> >>> line.
>> >>> Uncomment it if needed. Append the path to your cassandra sink and its
>> >>> associated jars, and see if that works.
>> >>>
>> >>> There doesn't appear to be a version incompatability; simply, the
>> >>> Cassandra jar is not found in the classpath. And the reason for that
>> >>> is it
>> >>> looks like you have a typo in your flume.conf.
>> >>>
>> >>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>> >>>
>> >>> should probably be
>> >>>
>> >>> com.btoddb.flume.sinks.cassandra.CassandraSink
>> >>>
>> >>> - Connor
>> >>>
>> >>>
>> >>> On Mon, Jan 14, 2013 at 5:57 PM, Jeff Bachtel <[EMAIL PROTECTED]>
>> >>> wrote:
>> >>>>
>> >>>> I have a build of Flume 1.4.0 on FreeBSD 8, installed into
>> >>>> /usr/local/flume . I manually created
>> >>>> /usr/local/flume/plugins.d/flume-ng-cassandra-sink and untarred
>> >>>> flume-ng-cassandra-sink-1.0.0-SNAPSHOT-dist.tar.gz (from a local
>> >>>> pull/build
>> >>>> of btoddb's sources on github), so I have a lib/ directory that's
>> >>>> completely
>> >>>> populated with both the sink plugin and any possible jarfiles it
>> >>>> might rely
>> >>>> on.
>> >>>>
>> >>>> I've likewise tried just copying
>> >>>> flume-ng-cassandra-sink-1.0.0-SNAPSHOT.jar to flume/lib/ for it to
>> >>>> get
>> >>>> picked up.
>> >>>>
>> >>>> When I have a CassandraSink define in flume.conf as such:
>> >>>> webserver.sinks.cassandraSink.type >> >>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink
>> >>>> webserver.sinks.cassandraSink.channel = memoryChannel
>> >>>> webserver.sinks.cassandraSink.hosts = localhost
>> >>>> webserver.sinks.cassandraSink.cluster-name = MyCluster
>> >>>> webserver.sinks.cassandraSink.keyspace-name = ApacheLogs
>> >>>> webserver.sinks.cassandraSink.records-coldfam = Requests

Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB