Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Displaying Map output in MapReduce


Copy link to this message
-
Re: Displaying Map output in MapReduce
If you set the number of reduce tasks to zero, the outputs of the mappers
will be sent directly to the OutputFormat. You can debug your map phase of a
job by disabling reduce and inspecting the mapper outputs, and then
re-enable the reducer after you've got the mapping part of the job running
correctly.

-- Aaron

On Fri, Jul 2, 2010 at 10:10 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Minor correction - port is 50075:
> http://10.32.56.159:50075/logs/userlogs/
>
> On Fri, Jul 2, 2010 at 12:32 AM, Alberto Luengo Cabanillas <
> [EMAIL PROTECTED]> wrote:
>
> > Hi! Supposing that you´re using a virtualized environment, you should be
> > able to get to JobTracker Web Admin Interface from http://
> > <your_virtualized_
> > machine_ip>:50070/logs/userlogs. In case you´re running Hadoop from
> inside
> > a
> > cluster frontend, use this machine´s IP. Inside this directory, it will
> > exist another one called "userlogs" where you will see all map and reduce
> > tasks launch by the JobTracker and the TaskTrackers. These tasks will
> look
> > like 'job_[m,r]_0_xxxxxx' (or similar). Inside of every one of them there
> > will be three subdirectories (stdout, stdlog and stderr) which will
> contain
> > all your debugging messages.
> > Inside my MapReduce development programs I use the Java *LOG* class in a
> > sentence like this:
> >
> > private static final Log LOG > > LogFactory.getLog(<your_class_name>.class.getName());
> > LOG.info("***********INSIDE MY REDUCER SETUP**********");
> >
> > Besides these messages, a normal "System.out.println(...)" will also
> appear
> > in the above directories.
> >
> > Hope it helps!
> >
> > 2010/7/2 abc xyz <[EMAIL PROTECTED]>
> >
> > > Hi folks,
> > >
> > > I want to just display the output of my mapper on console or want to
> log
> > it
> > > in
> > > some file. System.out.println and Log4j's logger are not displaying the
> > > output.
> > > How can I see the intermediate results?
> > >
> > > Cheers
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
> > --
> > Alberto
> >
>