Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> KafkaLog4jAppender with custom layout


Copy link to this message
-
KafkaLog4jAppender with custom layout
I'm having trouble getting the KafkaLog4JAppender to honor my custom Log4J layout. Here is a snippet form my log4j.xml
  <appender class="kafka.producer.KafkaLog4jAppender" name="kafka">
    <param name="Host" value="localhost"/>
    <param name="Port" value="9092"/>
    <param name="Topic" value="my-topic"/>
    <layout class="org.apache.log4j.PatternLayout">
      <param value="%d{ISO8601} %p %c{2} - %m%n" name="ConversionPattern"/>
    </layout>
  </appender>

  <logger name="my.package">
    <level value="INFO"/>
    <appender-ref ref="kafka"/>
  </logger>

I get the log messages in Kafka, but not with the property formatting. I checked out the Scala source and I see where it's checking for a layout:

 override def append(event: LoggingEvent)  {
    val message : String = if( this.layout == null) {
      event.getRenderedMessage
    }
    else this.layout.format(event)
    LogLog.debug("[" + new Date(event.getTimeStamp).toString + "]" + message)
    val messageData : ProducerData[String, String]       new ProducerData[String, String](topic, message)
    producer.send(messageData);
  }

I wonder if maybe it should be calling this.getLayout() instead just this.layout (maybe log4j is lazy loading things?) N.B., I've also tried the same config with a properties file to no avail.

Any help is appreciated.

-David
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB