Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Reduce phase hangs


+
Robin Verlangen 2012-11-08, 10:33
+
Mohammad Tariq 2012-11-08, 10:43
+
Robin Verlangen 2012-11-08, 10:47
+
Robin Verlangen 2012-11-08, 12:09
Copy link to this message
-
Re: Reduce phase hangs
Maybe you need more heap or that nodes's health is not fine. Or maybe the
data block on that node is causing the code to go in a very long loop.
There can be several reasons. It's tough to say anything confidently
without knowing the real issue.

Regards,
    Mohammad Tariq

On Thu, Nov 8, 2012 at 5:39 PM, Robin Verlangen <[EMAIL PROTECTED]> wrote:

> It actually seems that only 1 of the 24 reducers hangs at the "copy"
> phase. Any solutions for this?
>
> Best regards,
>
> Robin Verlangen
> *Software engineer*
> *
> *
> W http://www.robinverlangen.nl
> E [EMAIL PROTECTED]
>
> <http://goo.gl/Lt7BC>
>
> Disclaimer: The information contained in this message and attachments is
> intended solely for the attention and use of the named addressee and may be
> confidential. If you are not the intended recipient, you are reminded that
> the information remains the property of the sender. You must not use,
> disclose, distribute, copy, print or rely on this e-mail. If you have
> received this message in error, please contact the sender immediately and
> irrevocably delete this message and any copies.
>
>
>
> On Thu, Nov 8, 2012 at 11:47 AM, Robin Verlangen <[EMAIL PROTECTED]> wrote:
>
>> Nothing special over there. Most of the jobs complete after quite some
>> time. However it makes no sense to me that it takes that long. Its probably
>> just a couple of hundreds of megabytes.
>>
>> Best regards,
>>
>> Robin Verlangen
>> *Software engineer*
>> *
>> *
>> W http://www.robinverlangen.nl
>> E [EMAIL PROTECTED]
>>
>> <http://goo.gl/Lt7BC>
>>
>> Disclaimer: The information contained in this message and attachments is
>> intended solely for the attention and use of the named addressee and may be
>> confidential. If you are not the intended recipient, you are reminded that
>> the information remains the property of the sender. You must not use,
>> disclose, distribute, copy, print or rely on this e-mail. If you have
>> received this message in error, please contact the sender immediately and
>> irrevocably delete this message and any copies.
>>
>>
>>
>> On Thu, Nov 8, 2012 at 11:43 AM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>
>>> Hello Robin,
>>>
>>>         Look at the TaskTracker logs and see if you find anything
>>> interesting there.
>>>
>>> Regards,
>>>     Mohammad Tariq
>>>
>>>
>>>
>>> On Thu, Nov 8, 2012 at 4:03 PM, Robin Verlangen <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi there,
>>>>
>>>> It seems that some of my jobs hang in the reduce phase for a very long
>>>> time (for example, days). Is there anything I could tweak on? The query is
>>>> pretty simple, like:
>>>>
>>>> SELECT SUM(colA), to_date(colB) AS dt FROM table GROUP BY to_date(colB)
>>>> ORDER BY dt ASC;
>>>>
>>>> Best regards,
>>>>
>>>> Robin Verlangen
>>>> *Software engineer*
>>>> *
>>>> *
>>>> W http://www.robinverlangen.nl
>>>> E [EMAIL PROTECTED]
>>>>
>>>> <http://goo.gl/Lt7BC>
>>>>
>>>> Disclaimer: The information contained in this message and attachments
>>>> is intended solely for the attention and use of the named addressee and may
>>>> be confidential. If you are not the intended recipient, you are reminded
>>>> that the information remains the property of the sender. You must not use,
>>>> disclose, distribute, copy, print or rely on this e-mail. If you have
>>>> received this message in error, please contact the sender immediately and
>>>> irrevocably delete this message and any copies.
>>>>
>>>>
>>>
>>
>
+
Robin Verlangen 2012-11-08, 13:23
+
Mark Grover 2012-11-09, 07:19