Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> hadoop's attempt directory logs


Copy link to this message
-
Re: hadoop's attempt directory logs
The logger should set to INFO because the information are logged at
INFO level.  The increased severity level to ERROR filtered out all
log entries from task attempts.
Hope this helps.

regards,
Eric
On Thu, Jan 19, 2012 at 6:13 PM, Abhijit Dhar <[EMAIL PROTECTED]> wrote:
> I added this in my $HADOOP_CONF_DIR/log4j.properties
>
> log4j.appender.TLA=org.apache.hadoop.chukwa.inputtools.log4j.TaskLogAppender
> log4j.appender.TLA.recordType=CrawlerLog
> log4j.appender.TLA.taskId=${hadoop.tasklog.taskid}
> log4j.appender.TLA.layout=org.apache.log4j.PatternLayout
> log4j.appender.TLA.layout.ConversionPattern=%d{ISO8601} %p [%t]
> [%c{5}.%M()]:: %m%n
>
> But, it looks like when I add this to the rootlogger
> (log4j.rootLogger=ERROR, stdout, TLA), it stops going through even my stdout
> log appender.
> Any idea, what might be going on? (chukwa jar is on my classpath, and I
> don't see any ClassNotFound exception, so I think that part is good)
>
> Thanks,
>
>
>
> On Tue, Jan 17, 2012 at 6:03 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>
>> This can be setup by using TaskLogAppender that comes with Chukwa.
>> In $HADOOP_CONF_DIR/log4j.properties, modify the TaskLogAppender
>> configuration
>> to:
>>
>>
>> log4j.appender.TLA=org.apache.hadoop.chukwa.inputtools.log4j.TaskLogAppender
>>
>> Make sure chukwa-0.x.0-client.jar is in Hadoop class path, and restart
>> Hadoop.
>>
>> This should enable Task Attempt Logs to be sent through Chukwa data
>> collection system, and also log a copy locally on the task tracker
>> node.  Hope this is what you are looking for.
>>
>> regards,
>> Eric
>>
>> On Mon, Jan 16, 2012 at 7:20 PM, Abhijit Dhar <[EMAIL PROTECTED]>
>> wrote:
>> > Hi,
>> >
>> > Could you please point me how to stream the attempt logs that hadoop
>> > generates to chukwa, while at the same time saving them the way hadoop
>> > saves
>> > by default under attempt directories/stdout ?
>> >
>> > Thanks a lot
>> >
>> > --
>> > View this message in context:
>> > http://apache-chukwa.679492.n3.nabble.com/hadoop-s-attempt-directory-logs-tp3665088p3665088.html
>> > Sent from the Chukwa - Users mailing list archive at Nabble.com.
>
>
>
>
> --
> Абижит Дар
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB