Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> Deadlock happening in ChukwaDailyRollingFileAppender


+
Abhijit Dhar 2012-06-22, 16:50
+
Eric Yang 2012-06-24, 19:20
+
Abhijit Dhar 2012-06-25, 01:27
Copy link to this message
-
Re: Deadlock happening in ChukwaDailyRollingFileAppender
Hi Abhijit,

SocketAppender sends log entries as a full entry.  Chunk is created
per log entry (can be multiple lines).  There is no need to do
additional escaping.  SocketAdaptor should be configured in
initial_adaptors like:

add SocketAdaptor JobSummary 9098 0

This means to tag the log file collected through port 9098 as
JobSummary log file.
Hope this helps.

regards,
Eric

On Sun, Jun 24, 2012 at 6:27 PM, Abhijit Dhar <[EMAIL PROTECTED]> wrote:
> Does SocketAppender ecsape newlines? Like, log an exception with a stack
> trace(multiple lines) as a single log statement? And can it register an
> adapter automatically?
>
> On Jun 24, 2012 12:20 PM, "Eric Yang" <[EMAIL PROTECTED]> wrote:
>>
>> Hi Abhijit,
>>
>> The dead lock happens when there are too many registrations with the
>> agent and agent has exceeded the open file descriptor limit.  I would
>> recommend to use Log4J SocketAppender with combination of
>> SocketAdaptor.  It reduces the overhead of file offset tracking and
>> reduces the number of file descriptors used by the agent.  It works
>> better than using ChukwaDailyFileAppender and file adaptor
>> combination.
>>
>> regards,
>> Eric
>>
>> On Fri, Jun 22, 2012 at 9:50 AM, Abhijit Dhar <[EMAIL PROTECTED]>
>> wrote:
>> > Hi,
>> >
>> > I am getting a deadlock in ChukwaDailyRollingFileAppender. The thread
>> > dump
>> > is:
>> >
>> > Found one Java-level deadlock:
>> > ============================>> > "Thread-4":
>> >  waiting to lock monitor 0x00007f1970004818 (object 0x00000000cb6669e8,
>> > a
>> > java.lang.Class),
>> >  which is held by "Thread for syncLogs"
>> > "Thread for syncLogs":
>> >  waiting to lock monitor 0x00007f1970004ea8 (object 0x00000000d65c44b8,
>> > a
>> > org.apache.log4j.Logger),
>> >  which is held by "main"
>> > "main":
>> >  waiting to lock monitor 0x00007f1970003d98 (object 0x00000000d65c3e80,
>> > a
>> > org.apache.log4j.Logger),
>> >  which is held by "communication thread"
>> > "communication thread":
>> >  waiting to lock monitor 0x0000000041eba250 (object 0x00000000d6736ec0,
>> > a
>> > java.lang.Object),
>> >  which is held by "main"
>> >
>> > Java stack information for the threads listed above:
>> > ==================================================>> > "Thread-4":
>> >        at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:213)
>> >        - waiting to lock <0x00000000cb6669e8> (a java.lang.Class for
>> > org.apache.hadoop.mapred.TaskLog)
>> >        at org.apache.hadoop.mapred.Child$1.run(Child.java:75)
>> > "Thread for syncLogs":
>> >        at org.apache.log4j.Category.getAllAppenders(Category.java:409)
>> >        - waiting to lock <0x00000000d65c44b8> (a
>> > org.apache.log4j.Logger)
>> >        at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:218)
>> >        - locked <0x00000000cb6669e8> (a java.lang.Class for
>> > org.apache.hadoop.mapred.TaskLog)
>> >        at org.apache.hadoop.mapred.Child$2.run(Child.java:89)
>> > "main":
>> >        at org.apache.log4j.Category.callAppenders(Category.java:202)
>> >        - waiting to lock <0x00000000d65c3e80> (a
>> > org.apache.log4j.Logger)
>> >        at org.apache.log4j.Category.forcedLog(Category.java:388)
>> >        at org.apache.log4j.Category.warn(Category.java:1008)
>> >        at
>> >
>> > org.apache.hadoop.chukwa.datacollection.controller.ChukwaAgentController.addByName(ChukwaAgentController.java:267)
>> >        at
>> >
>> > org.apache.hadoop.chukwa.inputtools.log4j.ChukwaDailyRollingFileAppender.subAppend(ChukwaDailyRollingFileAppender.java:565)
>> >        - locked <0x00000000d6736ec0> (a java.lang.Object)
>> >        at
>> > org.apache.log4j.WriterAppender.append(WriterAppender.java:159)
>> >        at
>> > org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:230)
>> >        - locked <0x00000000d65b38c0> (a
>> > com.glassdoor.chukwa.inputtools.log4j.GDChukwaDailyRollingFileAppender)
>> >        at
>> >
>> > org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:65)
+
Abhijit Dhar 2012-07-06, 01:01
+
Eric Yang 2012-07-06, 05:21
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB