Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> MapReduce: Read the logs


Copy link to this message
-
MapReduce: Read the logs
Hi,

When I run a MapReduce job, the userlogs are stored on each Datanode.

Like: ~/hadoop_drive/hadoop/mapred/local/userlogs/job_201211011005_0018

Under this directory I have on subdirectory per job, like:
drwx------ 2 hadoop hadoop 4096 nov  5 12:46
attempt_201211011005_0018_m_000010_0
drwx------ 2 hadoop hadoop 4096 nov  5 12:46
attempt_201211011005_0018_m_000016_0
drwx------ 2 hadoop hadoop 4096 nov  5 12:46
attempt_201211011005_0018_m_000017_0
drwx------ 2 hadoop hadoop 4096 nov  5 12:46
attempt_201211011005_0018_m_000021_0

Now, I want to take a look at all the logs. Is there an easy way to
retrieve them? Or do I need to log in each single node and look at
each directory of each task?

I would like to put all the logs into a single file.

Thanks,

JM