Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - MapReduce: Read the logs


Copy link to this message
-
Re: MapReduce: Read the logs
Jean-Marc Spaggiari 2012-11-08, 02:07
Do you mean under http://servername:50030/jobhistoryhome.jsp ?

>From there I can see the local logs, or the job results/summary. But I
can't see the logs of the other nodes. If I want to see the logs, I
need to click on the job ID, then on the map successful tasks, then on
all the tasks one by one, then on view all log. Ouf. It's long.

On my MapReduce I'm doing that:
Logger.getLogger(TimestampBasedRowMover.class).info ("Moving " +
Bytes.toString(Bytes.tail(values.getRow(), values.getRow().length -
8)));

That's what I'm trying to see on my logs :(

2012/11/7, Binglin Chang <[EMAIL PROTECTED]>:
> Just watch logs in jobtracker & history UI isn't enough for you?
>
>
> On Thu, Nov 8, 2012 at 9:44 AM, Jean-Marc Spaggiari
> <[EMAIL PROTECTED]
>> wrote:
>
>> Hi,
>>
>> When I run a MapReduce job, the userlogs are stored on each Datanode.
>>
>> Like: ~/hadoop_drive/hadoop/mapred/local/userlogs/job_201211011005_0018
>>
>> Under this directory I have on subdirectory per job, like:
>> drwx------ 2 hadoop hadoop 4096 nov  5 12:46
>> attempt_201211011005_0018_m_000010_0
>> drwx------ 2 hadoop hadoop 4096 nov  5 12:46
>> attempt_201211011005_0018_m_000016_0
>> drwx------ 2 hadoop hadoop 4096 nov  5 12:46
>> attempt_201211011005_0018_m_000017_0
>> drwx------ 2 hadoop hadoop 4096 nov  5 12:46
>> attempt_201211011005_0018_m_000021_0
>>
>> Now, I want to take a look at all the logs. Is there an easy way to
>> retrieve them? Or do I need to log in each single node and look at
>> each directory of each task?
>>
>> I would like to put all the logs into a single file.
>>
>> Thanks,
>>
>> JM
>>
>