How do I change the heap size for the Name Node (and Secondary Name Node) without changing the heap size for the data nodes, job tracker, and task trackers?

Thanks,
Dave Shine
Sr. Software Engineer

1180 Celebration Blvd Suite 101
Celebration FL 34747
Office 321-939-5093
Mobile 407-314-0122
Fax 321-939-5604
[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
http://www.channelintelligence.com<http://www.channelintelligence.com/>

________________________________
The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message.

Hi,

On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine
<[EMAIL PROTECTED]> wrote:
> How do I change the heap size for the Name Node (and Secondary Name Node) without changing the heap size for the data nodes, job tracker, and task trackers?

Which version are you using. I am pretty certain that 0.21 and above
have HADOOP_NAMENODE_OPTS which can be used to set the heap size only
for NameNode. You can check if this option is defined in
hadoop-env.sh.

Thanks
Hemanth
>
> Thanks,
> Dave Shine
> Sr. Software Engineer
>
> 1180 Celebration Blvd Suite 101
> Celebration FL 34747
> Office 321-939-5093
> Mobile 407-314-0122
> Fax 321-939-5604
> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
> http://www.channelintelligence.com<http://www.channelintelligence.com/>
>
>
>
> ________________________________
> The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message.
>

I'm using 0.20.2.

I've seen references to HADOOP_NAMENODE_OPTS, but nothing that clearly explains how to set them.

Any help would be greatly appreciated.

Dave Shine

-----Original Message-----
From: Hemanth Yamijala [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, November 30, 2010 9:39 PM
To: [EMAIL PROTECTED]
Subject: Re: Configuring Heap Size for Name Node

Hi,

On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine
<[EMAIL PROTECTED]> wrote:
> How do I change the heap size for the Name Node (and Secondary Name Node) without changing the heap size for the data nodes, job tracker, and task trackers?

Which version are you using. I am pretty certain that 0.21 and above
have HADOOP_NAMENODE_OPTS which can be used to set the heap size only
for NameNode. You can check if this option is defined in
hadoop-env.sh.

Thanks
Hemanth
>
> Thanks,
> Dave Shine
> Sr. Software Engineer
>
> 1180 Celebration Blvd Suite 101
> Celebration FL 34747
> Office 321-939-5093
> Mobile 407-314-0122
> Fax 321-939-5604
> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
> http://www.channelintelligence.com<http://www.channelintelligence.com/>
>
>
>
> ________________________________
> The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message.
>

HADOOP_NAMENODE_OPTS="-Xmx500m" will set it to 500MB.  The "OPTS" here
refers to JVM options.  -Xmx is a common JVM option to set the maximum
heap.

-- Philip

On Wed, Dec 1, 2010 at 4:27 AM, Dave Shine
<[EMAIL PROTECTED]> wrote:
> I'm using 0.20.2.
>
> I've seen references to HADOOP_NAMENODE_OPTS, but nothing that clearly explains how to set them.
>
> Any help would be greatly appreciated.
>
> Dave Shine
>
> -----Original Message-----
> From: Hemanth Yamijala [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, November 30, 2010 9:39 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Configuring Heap Size for Name Node
>
> Hi,
>
> On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine
> <[EMAIL PROTECTED]> wrote:
>> How do I change the heap size for the Name Node (and Secondary Name Node) without changing the heap size for the data nodes, job tracker, and task trackers?
>
> Which version are you using. I am pretty certain that 0.21 and above
> have HADOOP_NAMENODE_OPTS which can be used to set the heap size only
> for NameNode. You can check if this option is defined in
> hadoop-env.sh.
>
> Thanks
> Hemanth
>>
>> Thanks,
>> Dave Shine
>> Sr. Software Engineer
>>
>> 1180 Celebration Blvd Suite 101
>> Celebration FL 34747
>> Office 321-939-5093
>> Mobile 407-314-0122
>> Fax 321-939-5604
>> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>> http://www.channelintelligence.com<http://www.channelintelligence.com/>
>>
>>
>>
>> ________________________________
>> The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message.
>>
>

JVM command-line parameters such as this are generally set in the
hadoop-env.sh file.

dbr

On 12/2/2010 1:11 AM, Philip Zeyliger wrote:
> HADOOP_NAMENODE_OPTS="-Xmx500m" will set it to 500MB.  The "OPTS" here
> refers to JVM options.  -Xmx is a common JVM option to set the maximum
> heap.
>
> -- Philip
>
> On Wed, Dec 1, 2010 at 4:27 AM, Dave Shine
> <[EMAIL PROTECTED]> wrote:
>> I'm using 0.20.2.
>>
>> I've seen references to HADOOP_NAMENODE_OPTS, but nothing that clearly explains how to set them.
>>
>> Any help would be greatly appreciated.
>>
>> Dave Shine
>>
>> -----Original Message-----
>> From: Hemanth Yamijala [mailto:[EMAIL PROTECTED]]
>> Sent: Tuesday, November 30, 2010 9:39 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring Heap Size for Name Node
>>
>> Hi,
>>
>> On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine
>> <[EMAIL PROTECTED]> wrote:
>>> How do I change the heap size for the Name Node (and Secondary Name Node) without changing the heap size for the data nodes, job tracker, and task trackers?
>> Which version are you using. I am pretty certain that 0.21 and above
>> have HADOOP_NAMENODE_OPTS which can be used to set the heap size only
>> for NameNode. You can check if this option is defined in
>> hadoop-env.sh.
>>
>> Thanks
>> Hemanth
>>> Thanks,
>>> Dave Shine
>>> Sr. Software Engineer
>>>
>>> 1180 Celebration Blvd Suite 101
>>> Celebration FL 34747
>>> Office 321-939-5093
>>> Mobile 407-314-0122
>>> Fax 321-939-5604
>>> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>>> http://www.channelintelligence.com<http://www.channelintelligence.com/>
>>>
>>>
>>>
>>> ________________________________
>>> The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message.
>>>

Phillip and David,

I got it to work.  Thanks for the feedback.

Dave

-----Original Message-----
From: David B. Ritch [mailto:[EMAIL PROTECTED]]
Sent: Friday, December 03, 2010 8:37 AM
To: [EMAIL PROTECTED]
Subject: Re: Configuring Heap Size for Name Node

JVM command-line parameters such as this are generally set in the
hadoop-env.sh file.

dbr

On 12/2/2010 1:11 AM, Philip Zeyliger wrote:
> HADOOP_NAMENODE_OPTS="-Xmx500m" will set it to 500MB.  The "OPTS" here
> refers to JVM options.  -Xmx is a common JVM option to set the maximum
> heap.
>
> -- Philip
>
> On Wed, Dec 1, 2010 at 4:27 AM, Dave Shine
> <[EMAIL PROTECTED]> wrote:
>> I'm using 0.20.2.
>>
>> I've seen references to HADOOP_NAMENODE_OPTS, but nothing that clearly explains how to set them.
>>
>> Any help would be greatly appreciated.
>>
>> Dave Shine
>>
>> -----Original Message-----
>> From: Hemanth Yamijala [mailto:[EMAIL PROTECTED]]
>> Sent: Tuesday, November 30, 2010 9:39 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Configuring Heap Size for Name Node
>>
>> Hi,
>>
>> On Wed, Dec 1, 2010 at 1:38 AM, Dave Shine
>> <[EMAIL PROTECTED]> wrote:
>>> How do I change the heap size for the Name Node (and Secondary Name Node) without changing the heap size for the data nodes, job tracker, and task trackers?
>> Which version are you using. I am pretty certain that 0.21 and above
>> have HADOOP_NAMENODE_OPTS which can be used to set the heap size only
>> for NameNode. You can check if this option is defined in
>> hadoop-env.sh.
>>
>> Thanks
>> Hemanth
>>> Thanks,
>>> Dave Shine
>>> Sr. Software Engineer
>>>
>>> 1180 Celebration Blvd Suite 101
>>> Celebration FL 34747
>>> Office 321-939-5093
>>> Mobile 407-314-0122
>>> Fax 321-939-5604
>>> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>>> http://www.channelintelligence.com<http://www.channelintelligence.com/>
>>>
>>>
>>>
>>> ________________________________
>>> The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message.
>>>