Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - viewing log messages from mapper and reducer.


Copy link to this message
-
viewing log messages from mapper and reducer.
jamal sasha 2013-01-16, 21:10
Hi..
   I am trying to run the following example:
http://hadoop.apache.org/docs/r0.17.0/mapred_tutorial.html#Example%3A+WordCount+v1.0
Except in the mapper and reduce class I am logging via
import org.apache.log4j.Logger;

In the WordCount class I add the follwoing line
private static final Logger LOG = Logger.getLogger(WordCount.class);

and then in the map and reduce methods
I add LOG.info("mapper");
LOG.info("reducer");

But those are not outputed in the console (I am running on pseudo
distributed mode)
In the userlogs directory there are four folders
log.index  stderr  stdout  syslog

In syslog, I have
2013-01-16 10:09:14,230 INFO org.apache.hadoop.util.NativeCodeLoader:
Loaded the native-hadoop library
2013-01-16 10:09:14,353 WARN
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
exists!
2013-01-16 10:09:14,414 INFO org.apache.hadoop.util.ProcessTree: setsid
exited with exit code 0
2013-01-16 10:09:14,418 INFO org.apache.hadoop.mapred.Task:  Using
ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@60cf710e
2013-01-16 10:09:14,489 INFO org.apache.hadoop.mapred.Task:
Task:attempt_201301021121_0131_m_000003_0 is done. And is in the process of
commiting
2013-01-16 10:09:17,411 INFO org.apache.hadoop.mapred.Task: Task
'attempt_201301021121_0131_m_000003_0' done.
2013-01-16 10:09:17,433 INFO org.apache.hadoop.mapred.TaskLogsTruncater:
Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2013-01-16 10:09:17,455 INFO org.apache.hadoop.io.nativeio.NativeIO:
Initialized cache for UID to User mapping with a cache timeout of 14400
seconds.
2013-01-16 10:09:17,456 INFO org.apache.hadoop.io.nativeio.NativeIO: Got
UserName mhduser for UID 1003 from the native implementation

Rest of the files are empty.. (except log.index)
Where is the logs being saved?
Thanks