Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - Logging in new clients


Copy link to this message
-
Re: Logging in new clients
Jay Kreps 2014-02-03, 22:02
Alright, I'm sold. If no objections, slf4j it is.

-Jay
On Mon, Feb 3, 2014 at 1:11 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:

> > I see server and client logging and dependencies are really different in
> my
> > mind. The server is a standalone process. Once we decouple the clients I
> > think we can actually be a lot more aggressive about using more libraries
> > on the server, why not? We have avoided that so far because they
> inevitably
> > leak into the clients since they arent separated.
> >
> > The problem on the client side is that if we choose anything used by our
> > clients and there is any incompatibility then they can't use kafka.
> >
> > From my point of view the improved logging libraries (log4j2, logback,
> etc)
> > are the worst option. They are new so likely will have api changes, and
> few
> > people use them so they don't get the benefit of sharing the same logging
> > library.
>
> Right - I was actually advocating sl4fj for this very reason (not a
> specific logging implementation).
>
> Clients would use the producer/consumer as part of some larger system
> that is more than likely to use something other than j.u.l (since no
> one seems to use it). With a layer like slf4j clients can choose
> whatever logging implementation they wish.  If we go with j.u.l then
> we would force all clients to include a j.u.l configuration file just
> for their producer/consumer in addition to the configuration file for
> the rest of their system.
>
> Thanks,
>
> Joel
>
> >
> > On Mon, Feb 3, 2014 at 11:17 AM, Joel Koshy <[EMAIL PROTECTED]> wrote:
> >
> > > We are already using other libraries in various parts of our code
> > > (e.g., metrics, zkclient, joptsimple, etc) some of which pull in these
> > > other logging dependencies anyway. i.e., what do we gain by using jul?
> > > There may be a good reason why people don't use jul so I think we
> > > should fully understand that before going with jul. So it may be
> > > simpler to just stick with log4j for the client rewrites and
> > > investigate logging later.
> > >
> > > log4j2 is becoming more widespread and many users seem to be favorable
> > > toward logback. slf4j would cover all of these very easily. From what
> > > I understand jul does not make it very easy to plug in with these
> > > various options but I could be wrong.
> > >
> > > I completely agree on the need to fix our client logging as that will
> > > go a long way in usability for end-users unless we want to keep
> > > getting asked the "Why do I see this ERROR in my logs..?" questions.
> > >
> > > Joel
> > >
> > > On Mon, Feb 03, 2014 at 11:08:39AM -0800, Neha Narkhede wrote:
> > > > >> Basically my preference would be java.util.logging unless there is
> > > some
> > > > known problem with it, otherwise I guess slf4j, and if not that then
> > > log4j.
> > > >
> > > > +1. My preference is to use java.util.logging to avoid adding an
> external
> > > > dependency,
> > > > but I'm not too sure about what's the "standard" out there, so open
> to
> > > > suggestions
> > > > on picking a different library.
> > > >
> > > >
> > > >
> > > > On Mon, Feb 3, 2014 at 11:00 AM, Jay Kreps <[EMAIL PROTECTED]>
> wrote:
> > > >
> > > > > We probably need to add a small amount of logging in the new
> producer
> > > and
> > > > > (soon) consumer clients. I wanted to have a quick discussion on
> logging
> > > > > libraries before I start adding this in the producer.
> > > > >
> > > > > Previously we have been pretty verbose loggers and I think we
> should
> > > stop
> > > > > that. For clients you mostly don't need to log: if there is an
> error
> > > you
> > > > > should throw it back not log it, so you don't need ERROR logging.
> > > Likewise
> > > > > I think it is rude to pollute peoples logs with the details of
> client
> > > > > initialization (establishing connections, etc), so you don't need
> INFO
> > > > > logging. However perhaps there is an argument to be made for WARN
> and
> > > > > DEBUG. I think it is perhaps useful to log a WARN when a server