Hello,
I want to set more heap space to my scripts, but I can't make Pig
support this, when I call: "pig -Dmapred.child.java.opts=-Xmx2048"
It fails (just prints help), and option --help doesn't show any
information about -D parameter.

How can I do this?
I'm using version 0.7.0+9.
--
Wojciech Langiewicz

Wojcieh,

One alternative is setting this and other parameters in a
mapred-site.xml file in your classpath.

Chhers,
Marcos

> Hello,
> I want to set more heap space to my scripts, but I can't make Pig
> support this, when I call: "pig -Dmapred.child.java.opts=-Xmx2048"
> It fails (just prints help), and option --help doesn't show any
> information about -D parameter.
>
> How can I do this?
> I'm using version 0.7.0+9.
> --
> Wojciech Langiewicz
>

Hi,
But will this work with different settings for every job?
Because this is what I essentially need.

Marcos Medrado Rubinelli wrote:
> Wojcieh,
>
> One alternative is setting this and other parameters in a
> mapred-site.xml file in your classpath.
>
> Chhers,
> Marcos
>
>> Hello,
>> I want to set more heap space to my scripts, but I can't make Pig
>> support this, when I call: "pig -Dmapred.child.java.opts=-Xmx2048"
>> It fails (just prints help), and option --help doesn't show any
>> information about -D parameter.
>>
>> How can I do this?
>> I'm using version 0.7.0+9.
>> --
>> Wojciech Langiewicz

Hi Wojciech,

Well, the parameter mapred.child.java.opts is also subject to memory limit of your box and by the limit set by mapred.child.ulimit.
It is ideal though to find the root cause of heap error. If needed use specific mapred.(map|reduce).child.java.opts as needed.

In some cases it needs to be rectified by using nocombiner (-Dpig.exec.nocombiner=true) but I reckon that would be 0.8..

Also as Olga confirmed from 0.8 you would be able to see -D params list settable from pig command  using -help -properties

Thanks & Regards,
/Rekha

On 10/25/10 2:15 AM, "Wojciech Langiewicz" <[EMAIL PROTECTED]> wrote:

Hi,
But will this work with different settings for every job?
Because this is what I essentially need.

Marcos Medrado Rubinelli wrote:
> Wojcieh,
>
> One alternative is setting this and other parameters in a
> mapred-site.xml file in your classpath.
>
> Chhers,
> Marcos
>
>> Hello,
>> I want to set more heap space to my scripts, but I can't make Pig
>> support this, when I call: "pig -Dmapred.child.java.opts=-Xmx2048"
>> It fails (just prints help), and option --help doesn't show any
>> information about -D parameter.
>>
>> How can I do this?
>> I'm using version 0.7.0+9.
>> --
>> Wojciech Langiewicz

If you want to externally specify properties for *a* pig script you
can put the key value pairs in a conf file and pass its location using
--config  to pig and those properties will be included.

If you want to externally specify properties for *all* pig script that
you run, then modify conf/pig.properties by putting in your key value
pairs in it.

If you want to specify properties from *within* a pig script then you
can use set command in the pig script. This feature will be there in
upcoming 0.8 release.

Hope it helps,
Ashutosh

On Sun, Oct 24, 2010 at 13:45, Wojciech Langiewicz
<[EMAIL PROTECTED]> wrote:
> Hi,
> But will this work with different settings for every job?
> Because this is what I essentially need.
>
> Marcos Medrado Rubinelli wrote:
>>
>> Wojcieh,
>>
>> One alternative is setting this and other parameters in a
>> mapred-site.xml file in your classpath.
>>
>> Chhers,
>> Marcos
>>
>>> Hello,
>>> I want to set more heap space to my scripts, but I can't make Pig
>>> support this, when I call: "pig -Dmapred.child.java.opts=-Xmx2048"
>>> It fails (just prints help), and option --help doesn't show any
>>> information about -D parameter.
>>>
>>> How can I do this?
>>> I'm using version 0.7.0+9.
>>> --
>>> Wojciech Langiewicz
>
>

 The way I get pig options is:

PIG_OPTS="$PIG_OPTS
-Dmapred.cache.archives=hdfs://server:8020/pat/data.$date#index "
export PIG_OPTS
pig ./Piggy.pig

Allan

-----Original Message-----
From: Ashutosh Chauhan [mailto:[EMAIL PROTECTED]]
Sent: Sunday, October 24, 2010 6:43 PM
To: [EMAIL PROTECTED]
Subject: Re: Passing Java options to pig scripts

If you want to externally specify properties for *a* pig script you can
put the key value pairs in a conf file and pass its location using
--config  to pig and those properties will be included.

If you want to externally specify properties for *all* pig script that
you run, then modify conf/pig.properties by putting in your key value
pairs in it.

If you want to specify properties from *within* a pig script then you
can use set command in the pig script. This feature will be there in
upcoming 0.8 release.

Hope it helps,
Ashutosh

On Sun, Oct 24, 2010 at 13:45, Wojciech Langiewicz
<[EMAIL PROTECTED]> wrote:
> Hi,
> But will this work with different settings for every job?
> Because this is what I essentially need.
>
> Marcos Medrado Rubinelli wrote:
>>
>> Wojcieh,
>>
>> One alternative is setting this and other parameters in a
>> mapred-site.xml file in your classpath.
>>
>> Chhers,
>> Marcos
>>
>>> Hello,
>>> I want to set more heap space to my scripts, but I can't make Pig
>>> support this, when I call: "pig -Dmapred.child.java.opts=-Xmx2048"
>>> It fails (just prints help), and option --help doesn't show any
>>> information about -D parameter.
>>>
>>> How can I do this?
>>> I'm using version 0.7.0+9.
>>> --
>>> Wojciech Langiewicz
>
>