Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka's agent sit on localhost to forward message to broker?


Copy link to this message
-
Re: Kafka's agent sit on localhost to forward message to broker?
Yeah, the console producer should be able to do this out of the box.

The only concern would be to make sure log rotation is handled properly,
but I'm sure that wouldn't be too hard...

--
Felix

On Wed, Oct 10, 2012 at 1:15 PM, Zsolt Dollenstein <[EMAIL PROTECTED]>wrote:

> You mean the producer?
>
> On Wed, Oct 10, 2012 at 5:23 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> > Yes, perhaps the existing console consumer could be modified to do this?
> >
> > -Jay
> >
> > On Tue, Oct 9, 2012 at 10:05 PM, howard chen <[EMAIL PROTECTED]> wrote:
> >
> >> Hi,
> >>
> >> On Wed, Oct 10, 2012 at 12:20 AM, Jay Kreps <[EMAIL PROTECTED]>
> wrote:
> >> > I am not aware of anyone having open sourced what you describe. If
> anyone
> >> > is interested one nice way to implement this would be to implement a
> >> daemon
> >> > that could interact with the syslogd protocol over a unix domain
> socket.
> >> > Many scripting languages have good syslogd integration and this would
> >> > probably be a good strategy for supporting logging from this kind of
> >> > process.
> >> >
> >>
> >> In the simplest way, able to tail a file would be enough for this
> purpose.
> >>
> >> Existing app keep writing to local log file, no any blocking and not
> >> affected by network connection, and another process tail the file and
> >> send the changes to broker.
> >>
> >> Scribe and Flume also have this build in, would be nice if Kafka also
> >> support this out of the box.
> >>
> >> (From old posts it was said that Window does not have tail...well, I
> >> think most people run on Linux anyway..)
> >>
> >> Thanks.
> >>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB