Hi,

How do I change the default timeout for reducer with Pig? I have some
reducer that needs to take longer than 10 minutes to finish. It is pretty
frustrating to see many of get to 95% complete and then got killed.

Thanks.
Dexin

"set mapred.task.timeout $num_milliseconds"

On Tue, Dec 21, 2010 at 1:23 PM, Dexin Wang <[EMAIL PROTECTED]> wrote:

> Hi,
>
> How do I change the default timeout for reducer with Pig? I have some
> reducer that needs to take longer than 10 minutes to finish. It is pretty
> frustrating to see many of get to 95% complete and then got killed.
>
> Thanks.
> Dexin
>

Ideally you need not to do that. Pig automatically takes care of
progress reporting in its operator. Do you have a pig script which
fails because of reporting progress timeout issues ?

Ashutosh

On Tue, Dec 21, 2010 at 13:23, Dexin Wang <[EMAIL PROTECTED]> wrote:
> Hi,
>
> How do I change the default timeout for reducer with Pig? I have some
> reducer that needs to take longer than 10 minutes to finish. It is pretty
> frustrating to see many of get to 95% complete and then got killed.
>
> Thanks.
> Dexin
>

True, however there is one bug in 0.7. We fix it in 0.8.

https://issues.apache.org/jira/browse/PIG-1760

Daniel
Ashutosh Chauhan wrote:
> Ideally you need not to do that. Pig automatically takes care of
> progress reporting in its operator. Do you have a pig script which
> fails because of reporting progress timeout issues ?
>
> Ashutosh
>
> On Tue, Dec 21, 2010 at 13:23, Dexin Wang <[EMAIL PROTECTED]> wrote:
>  
>> Hi,
>>
>> How do I change the default timeout for reducer with Pig? I have some
>> reducer that needs to take longer than 10 minutes to finish. It is pretty
>> frustrating to see many of get to 95% complete and then got killed.
>>
>> Thanks.
>> Dexin
>>
>>

It doesn't seem to work. I got

2010-12-22 21:36:59,120 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1000: Error during parsing. Unrecognized set key: mapred.task.timeout

I did this in my pig script:

SET mapred.task.timeout 1800000;

What did I do wrong? Thanks.

% pig --version
Apache Pig version 0.7.0+9 (rexported)
compiled Jun 28 2010, 12:53:50

On Tue, Dec 21, 2010 at 2:39 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> True, however there is one bug in 0.7. We fix it in 0.8.
>
> https://issues.apache.org/jira/browse/PIG-1760
>
> Daniel
>
>
>
> Ashutosh Chauhan wrote:
>
>> Ideally you need not to do that. Pig automatically takes care of
>> progress reporting in its operator. Do you have a pig script which
>> fails because of reporting progress timeout issues ?
>>
>> Ashutosh
>>
>> On Tue, Dec 21, 2010 at 13:23, Dexin Wang <[EMAIL PROTECTED]> wrote:
>>
>>
>>> Hi,
>>>
>>> How do I change the default timeout for reducer with Pig? I have some
>>> reducer that needs to take longer than 10 minutes to finish. It is pretty
>>> frustrating to see many of get to 95% complete and then got killed.
>>>
>>> Thanks.
>>> Dexin
>>>
>>>
>>>
>>
>

SET is only available in 0.8 not in 0.7. But I think your problem lies
elsewhere and most likely is not a reporting timeout issue.

Ashutosh
On Wednesday, December 22, 2010, Dexin Wang <[EMAIL PROTECTED]> wrote:
> It doesn't seem to work. I got
>
> 2010-12-22 21:36:59,120 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1000: Error during parsing. Unrecognized set key: mapred.task.timeout
>
> I did this in my pig script:
>
> SET mapred.task.timeout 1800000;
>
> What did I do wrong? Thanks.
>
> % pig --version
> Apache Pig version 0.7.0+9 (rexported)
> compiled Jun 28 2010, 12:53:50
>
> On Tue, Dec 21, 2010 at 2:39 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
>
>> True, however there is one bug in 0.7. We fix it in 0.8.
>>
>> https://issues.apache.org/jira/browse/PIG-1760
>>
>> Daniel
>>
>>
>>
>> Ashutosh Chauhan wrote:
>>
>>> Ideally you need not to do that. Pig automatically takes care of
>>> progress reporting in its operator. Do you have a pig script which
>>> fails because of reporting progress timeout issues ?
>>>
>>> Ashutosh
>>>
>>> On Tue, Dec 21, 2010 at 13:23, Dexin Wang <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>>> Hi,
>>>>
>>>> How do I change the default timeout for reducer with Pig? I have some
>>>> reducer that needs to take longer than 10 minutes to finish. It is pretty
>>>> frustrating to see many of get to 95% complete and then got killed.
>>>>
>>>> Thanks.
>>>> Dexin
>>>>
>>>>
>>>>
>>>
>>
>