Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - set reducer timeout with pig


Copy link to this message
-
Re: set reducer timeout with pig
Dexin Wang 2010-12-23, 03:38
It doesn't seem to work. I got

2010-12-22 21:36:59,120 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1000: Error during parsing. Unrecognized set key: mapred.task.timeout

I did this in my pig script:

SET mapred.task.timeout 1800000;

What did I do wrong? Thanks.

% pig --version
Apache Pig version 0.7.0+9 (rexported)
compiled Jun 28 2010, 12:53:50

On Tue, Dec 21, 2010 at 2:39 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> True, however there is one bug in 0.7. We fix it in 0.8.
>
> https://issues.apache.org/jira/browse/PIG-1760
>
> Daniel
>
>
>
> Ashutosh Chauhan wrote:
>
>> Ideally you need not to do that. Pig automatically takes care of
>> progress reporting in its operator. Do you have a pig script which
>> fails because of reporting progress timeout issues ?
>>
>> Ashutosh
>>
>> On Tue, Dec 21, 2010 at 13:23, Dexin Wang <[EMAIL PROTECTED]> wrote:
>>
>>
>>> Hi,
>>>
>>> How do I change the default timeout for reducer with Pig? I have some
>>> reducer that needs to take longer than 10 minutes to finish. It is pretty
>>> frustrating to see many of get to 95% complete and then got killed.
>>>
>>> Thanks.
>>> Dexin
>>>
>>>
>>>
>>
>