Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - Displaying Map output in MapReduce


Copy link to this message
-
Re: Displaying Map output in MapReduce
Aaron Kimball 2010-07-05, 07:34
If you set the number of reduce tasks to zero, the outputs of the mappers
will be sent directly to the OutputFormat. You can debug your map phase of a
job by disabling reduce and inspecting the mapper outputs, and then
re-enable the reducer after you've got the mapping part of the job running
correctly.

-- Aaron

On Fri, Jul 2, 2010 at 10:10 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Minor correction - port is 50075:
> http://10.32.56.159:50075/logs/userlogs/
>
> On Fri, Jul 2, 2010 at 12:32 AM, Alberto Luengo Cabanillas <
> [EMAIL PROTECTED]> wrote:
>
> > Hi! Supposing that you´re using a virtualized environment, you should be
> > able to get to JobTracker Web Admin Interface from http://
> > <your_virtualized_
> > machine_ip>:50070/logs/userlogs. In case you´re running Hadoop from
> inside
> > a
> > cluster frontend, use this machine´s IP. Inside this directory, it will
> > exist another one called "userlogs" where you will see all map and reduce
> > tasks launch by the JobTracker and the TaskTrackers. These tasks will
> look
> > like 'job_[m,r]_0_xxxxxx' (or similar). Inside of every one of them there
> > will be three subdirectories (stdout, stdlog and stderr) which will
> contain
> > all your debugging messages.
> > Inside my MapReduce development programs I use the Java *LOG* class in a
> > sentence like this:
> >
> > private static final Log LOG > > LogFactory.getLog(<your_class_name>.class.getName());
> > LOG.info("***********INSIDE MY REDUCER SETUP**********");
> >
> > Besides these messages, a normal "System.out.println(...)" will also
> appear
> > in the above directories.
> >
> > Hope it helps!
> >
> > 2010/7/2 abc xyz <[EMAIL PROTECTED]>
> >
> > > Hi folks,
> > >
> > > I want to just display the output of my mapper on console or want to
> log
> > it
> > > in
> > > some file. System.out.println and Log4j's logger are not displaying the
> > > output.
> > > How can I see the intermediate results?
> > >
> > > Cheers
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > --
> > Alberto
> >
>