Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Kafka's agent sit on localhost to forward message to broker?


+
howard chen 2012-10-09, 15:48
+
Neha Narkhede 2012-10-09, 15:51
+
howard chen 2012-10-09, 15:53
+
Jay Kreps 2012-10-09, 16:20
+
howard chen 2012-10-10, 05:05
+
Jay Kreps 2012-10-10, 15:23
+
Zsolt Dollenstein 2012-10-10, 17:15
Copy link to this message
-
Re: Kafka's agent sit on localhost to forward message to broker?
Felix GV 2012-10-18, 20:31
Yeah, the console producer should be able to do this out of the box.

The only concern would be to make sure log rotation is handled properly,
but I'm sure that wouldn't be too hard...

--
Felix

On Wed, Oct 10, 2012 at 1:15 PM, Zsolt Dollenstein <[EMAIL PROTECTED]>wrote:

> You mean the producer?
>
> On Wed, Oct 10, 2012 at 5:23 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> > Yes, perhaps the existing console consumer could be modified to do this?
> >
> > -Jay
> >
> > On Tue, Oct 9, 2012 at 10:05 PM, howard chen <[EMAIL PROTECTED]> wrote:
> >
> >> Hi,
> >>
> >> On Wed, Oct 10, 2012 at 12:20 AM, Jay Kreps <[EMAIL PROTECTED]>
> wrote:
> >> > I am not aware of anyone having open sourced what you describe. If
> anyone
> >> > is interested one nice way to implement this would be to implement a
> >> daemon
> >> > that could interact with the syslogd protocol over a unix domain
> socket.
> >> > Many scripting languages have good syslogd integration and this would
> >> > probably be a good strategy for supporting logging from this kind of
> >> > process.
> >> >
> >>
> >> In the simplest way, able to tail a file would be enough for this
> purpose.
> >>
> >> Existing app keep writing to local log file, no any blocking and not
> >> affected by network connection, and another process tail the file and
> >> send the changes to broker.
> >>
> >> Scribe and Flume also have this build in, would be nice if Kafka also
> >> support this out of the box.
> >>
> >> (From old posts it was said that Window does not have tail...well, I
> >> think most people run on Linux anyway..)
> >>
> >> Thanks.
> >>
>
+
David Birdsong 2012-10-19, 23:27