Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - flume-cassandra


+
Sri Ramya 2013-01-23, 17:59
+
rektide 2013-01-23, 18:23
+
Camp, Roy 2013-01-23, 18:52
+
shekhar sharma 2013-01-24, 01:53
+
Sri Ramya 2013-01-24, 03:30
+
Mike Percy 2013-01-24, 04:56
+
Sri Ramya 2013-01-24, 05:05
+
Mike Percy 2013-01-24, 05:13
+
Sri Ramya 2013-01-24, 05:45
+
Mike Percy 2013-01-24, 08:25
+
Priyanka Jain 2013-01-24, 05:46
+
Sri Ramya 2013-01-24, 05:50
+
Ralph Goers 2013-01-28, 16:56
+
Sri Ramya 2013-01-29, 03:54
+
Sri Ramya 2013-01-29, 05:15
+
Alexander Alten-Lorenz 2013-01-29, 07:31
+
Ralph Goers 2013-01-29, 19:24
+
Yogi Nerella 2013-01-29, 19:43
+
Ralph Goers 2013-01-29, 21:59
Copy link to this message
-
Re: flume-cassandra
Yogi Nerella 2013-01-29, 22:07
Simply superb, appreciate your help for taking time to answer all my
questions.

Thanks again
Yogi
On Tue, Jan 29, 2013 at 1:59 PM, Ralph Goers <[EMAIL PROTECTED]>wrote:

> See below.
>
> On Jan 29, 2013, at 11:43 AM, Yogi Nerella wrote:
>
> Ralph,
>
> Sorry Ramya for side tracking this thread.
>
> I am also looking for some open source tools to collect all log records
> from various applications into a central place.
> Looking at Cassandra File System as well, but really do not understand
> what value we get by storing in it, instead of storing in NFS based file
> system.
>
>
> Cassandra is not a file system. It is a NoSQL solution that stores the
> data elements as columns in rows.  See
> http://nosql.mypopescu.com/post/2981945438/why-netflix-picked-amazon-simpledb-hadoop-hbase-and for
> a nice discussion on the differences between Hadoop, Cassandra and SimpleDB.
>
>
>
> Are there any good UI tools for searching the CFS database for
> administrators to look at errors?
>
>
> No.  That is similar to asking if there are good tools to looking for
> "xxxx" in database "yyyy" where "xxxx" is something specific to your
> problem.   As a NoSQL database Cassandra is not solving a specific problem
> such as capturing logs.
>
>
> Is MDC working with flume log4jappender?   I couldnt make it work?
>
>
> Flume's Log4j appender is for Log4j 1.x.  I am working on Log4j 2 which is
> a complete rewrite and is designed for modern JDKs and to be able to
> reliably handle audit logging. See
> http://logging.apache.org/log4j/2.x/manual/appenders.html#FlumeAppender.
>  Log4j 2 supports both writing to a remote agent over Avro or an embedded
> agent.
>
>
> log4jappender of flume has so many dependencies, are there any good
> "SocketAppenders", which can store and forward in the case of the server
> down?
>
>
> One of the unfortunate aspects of Flume is that it does have a lot of
> dependencies. When you use the Avro remote Appender you will only need a
> few jars from Log4j,  Flume, Avro and possibly a couple more that they
> depend on. If you use the embedded appender you will need quite a few more
> since you are essentially running a Flume agent in your application.
>
> Generic socket appenders generally suffer from the problem that they
> cannot guarantee delivery.  IOW, you get control back as soon as TCP says
> the data was delivered. It might never make it into the Flume channel.
>  When you use Avro you do get guaranteed delivery as you don't get control
> back until the data is written to the Flume channel. A "Socket Appender"
> that can store and forward would be the Flume embedded appender in Log4j 2.
>
>
> Can I use "SocketAppender" and send the data to "flume agent" configured
> to receive netcat messages?
>
>
> Yes, you can use a Socket Appender.  Unlike other logging frameworks, with
> Log4j 2 you can use a Layout to format the data going over the socket
> connection in any manner you want.
>
> Ralph
>
+
Sri Ramya 2013-01-31, 04:35
+
Ralph Goers 2013-01-31, 07:10
+
Sri Ramya 2013-01-31, 10:56
+
Sri Ramya 2013-02-01, 14:32