Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: reducer not starting


Copy link to this message
-
Re: reducer not starting
As harsh suggested, you might want to check the task logs on slaves (you
can do it though web UI by clicking on map/reduce task links) and see if
there are any exceptions .
On Wed, Nov 21, 2012 at 8:06 PM, jamal sasha <[EMAIL PROTECTED]> wrote:

> Hi
>   Thanks for the insights.
> I noticed that these restarts of mappers were because in the shebang i had
> Usr/env/bin instead of usr/env/bin python
> Any clue of what was going on with reducers not starting but mappers being
> executed again and again.
> Probably a very naive question but i am newbie you see :)
>
>
>
> On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
> > Just FYI, you don't need to stop the job, update the host, and retry.
> >
> > Just update the host while the job is running and it should retry and
> restart.
> >
> > I had a similar issue with one of my node where the hosts file were
> > not updated. After the updated it has automatically resume the work...
> >
> > JM
> >
> > 2012/11/21, praveenesh kumar <[EMAIL PROTECTED]>:
> >> Sometimes its network issue, reducers are not able to find hostnames or
> IPs
> >> of the other machines. Make sure your /etc/hosts entries and hostnames
> are
> >> correct.
> >>
> >> Regards,
> >> Praveenesh
> >>
> >> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> >>
> >>> Your mappers are failing (possibly a user-side error or an
> >>> environmental one) and are being reattempted by the framework (default
> >>> behavior, attempts 4 times to avoid transient failure scenario).
> >>>
> >>> Visit your job's logs in the JobTracker web UI, to find more
> >>> information on why your tasks fail.
> >>>
> >>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <[EMAIL PROTECTED]>
> >>> wrote:
> >>> >
> >>> >
> >>> >
> >>> > I am not sure whats happening, but I wrote a simple mapper and
> reducer
> >>> > script.
> >>> >
> >>> >
> >>> >
> >>> > And I am testing it against a small dataset (like few lines long).
> >>> >
> >>> >
> >>> >
> >>> > For some reason reducer is just not starting.. and mapper is
> executing
> >>> again
> >>> > and again?
> >>> >
> >>> >
> >>> >
> >>> >     12/11/20 09:21:18 INFO streaming.StreamJob:  map 0%  reduce 0%
> >>> >
> >>> >     12/11/20 09:22:05 INFO streaming.StreamJob:  map 50%  reduce 0%
> >>> >
> >>> >     12/11/20 09:22:10 INFO streaming.StreamJob:  map 100%  reduce 0%
> >>> >
> >>> >     12/11/20 09:32:05 INFO streaming.StreamJob:  map 50%  reduce 0%
> >>> >
> >>> >     12/11/20 09:32:11 INFO streaming.StreamJob:  map 0%  reduce 0%
> >>> >
> >>> >     12/11/20 09:32:20 INFO streaming.StreamJob:  map 50%  reduce 0%
> >>> >
> >>> >     12/11/20 09:32:31 INFO streaming.StreamJob:  map 100%  reduce 0%
> >>> >
> >>> >     12/11/20 09:42:20 INFO streaming.StreamJob:  map 50%  reduce 0%
> >>> >
> >>> >     12/11/20 09:42:31 INFO streaming.StreamJob:  map 0%  reduce 0%
> >>> >
> >>> >     12/11/20 09:42:32 INFO streaming.StreamJob:  map 50%  reduce 0%
> >>> >
> >>> >     12/11/20 09:42:50 INFO streaming.StreamJob:  map 100%  reduce 0%
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > Let me know if you want the code also.
> >>> >
> >>> > Any clues of where I am going wrong?
> >>> >
> >>> > Thanks
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>>
> >>
> >
>

--
Regards,
Bharath .V
w:http://researchweb.iiit.ac.in/~bharath.v