Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - reducer not starting


+
jamal sasha 2012-11-20, 16:52
+
Harsh J 2012-11-20, 17:16
+
praveenesh kumar 2012-11-21, 12:00
+
Jean-Marc Spaggiari 2012-11-21, 12:45
Copy link to this message
-
Re: reducer not starting
jamal sasha 2012-11-21, 14:36
Hi
  Thanks for the insights.
I noticed that these restarts of mappers were because in the shebang i had
Usr/env/bin instead of usr/env/bin python
Any clue of what was going on with reducers not starting but mappers being
executed again and again.
Probably a very naive question but i am newbie you see :)
On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:
> Just FYI, you don't need to stop the job, update the host, and retry.
>
> Just update the host while the job is running and it should retry and
restart.
>
> I had a similar issue with one of my node where the hosts file were
> not updated. After the updated it has automatically resume the work...
>
> JM
>
> 2012/11/21, praveenesh kumar <[EMAIL PROTECTED]>:
>> Sometimes its network issue, reducers are not able to find hostnames or
IPs
>> of the other machines. Make sure your /etc/hosts entries and hostnames
are
>> correct.
>>
>> Regards,
>> Praveenesh
>>
>> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>>> Your mappers are failing (possibly a user-side error or an
>>> environmental one) and are being reattempted by the framework (default
>>> behavior, attempts 4 times to avoid transient failure scenario).
>>>
>>> Visit your job's logs in the JobTracker web UI, to find more
>>> information on why your tasks fail.
>>>
>>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <[EMAIL PROTECTED]>
>>> wrote:
>>> >
>>> >
>>> >
>>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>>> > script.
>>> >
>>> >
>>> >
>>> > And I am testing it against a small dataset (like few lines long).
>>> >
>>> >
>>> >
>>> > For some reason reducer is just not starting.. and mapper is executing
>>> again
>>> > and again?
>>> >
>>> >
>>> >
>>> >     12/11/20 09:21:18 INFO streaming.StreamJob:  map 0%  reduce 0%
>>> >
>>> >     12/11/20 09:22:05 INFO streaming.StreamJob:  map 50%  reduce 0%
>>> >
>>> >     12/11/20 09:22:10 INFO streaming.StreamJob:  map 100%  reduce 0%
>>> >
>>> >     12/11/20 09:32:05 INFO streaming.StreamJob:  map 50%  reduce 0%
>>> >
>>> >     12/11/20 09:32:11 INFO streaming.StreamJob:  map 0%  reduce 0%
>>> >
>>> >     12/11/20 09:32:20 INFO streaming.StreamJob:  map 50%  reduce 0%
>>> >
>>> >     12/11/20 09:32:31 INFO streaming.StreamJob:  map 100%  reduce 0%
>>> >
>>> >     12/11/20 09:42:20 INFO streaming.StreamJob:  map 50%  reduce 0%
>>> >
>>> >     12/11/20 09:42:31 INFO streaming.StreamJob:  map 0%  reduce 0%
>>> >
>>> >     12/11/20 09:42:32 INFO streaming.StreamJob:  map 50%  reduce 0%
>>> >
>>> >     12/11/20 09:42:50 INFO streaming.StreamJob:  map 100%  reduce 0%
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Let me know if you want the code also.
>>> >
>>> > Any clues of where I am going wrong?
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>