Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Java Heap space Exception


+
unmesha sreeveni 2013-11-15, 09:23
+
unmesha sreeveni 2013-11-15, 11:03
+
Rob Blah 2013-11-15, 13:16
+
Rob Blah 2013-11-15, 13:24
Copy link to this message
-
Re: Java Heap space Exception
Thx Rob Blah
On Fri, Nov 15, 2013 at 6:54 PM, Rob Blah <[EMAIL PROTECTED]> wrote:

> MapReduce
> http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
> "We’ll thus assign 4 GB for Map task Containers, and 8 GB for Reduce tasks
> Containers."
>
> You can try editing configuration file:
> <name>mapreduce.map.memory.mb</name>
> <value>?</value>
>
> <name>mapreduce.reduce.memory.mb</name>
>  <value>?</value>
>
> Or set these in a driver.
>
> If you have custom AM, try this:
> Driver:
> capability.setMemory(amMemory);
> appContext.setResource(capability);
>
> AM:
> Resource capability = Records.newRecord(Resource.class);
> capability.setMemory(containerMemory);
> AMRMClient.ContainerRequest request = new
> AMRMClient.ContainerRequest(capability, null, null, pri);
> OR AM (use java command to set heap size):
> vargs.add("-Xmx" + containerMemory + "m");
>
>
>
>
>
>
> 2013/11/15 Rob Blah <[EMAIL PROTECTED]>
>
>> Question have you tried giving more memory to the containers?
>>
>>
>> 2013/11/15 unmesha sreeveni <[EMAIL PROTECTED]>
>>
>>> ys found
>>> tried -D mapred.child.java.opts=-Xmx4096M on the command line:
>>>
>>>
>>> On Fri, Nov 15, 2013 at 3:01 PM, unmesha sreeveni <[EMAIL PROTECTED]
>>> > wrote:
>>>
>>>> hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata /user/unmesha
>>>> /out
>>>>
>>>>
>>>>
>>>> On Fri, Nov 15, 2013 at 2:53 PM, unmesha sreeveni <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> When i tried to execute my program with 100 MB file i am getting
>>>>> JavaHeapSpace Exception
>>>>>
>>>>>
>>>>> >hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata
>>>>> /user/inputdata/out
>>>>>
>>>>> How to increase the heap size through commandline
>>>>>
>>>>>
>>>>> --
>>>>> *Thanks & Regards*
>>>>>
>>>>> Unmesha Sreeveni U.B
>>>>>
>>>>> *Junior Developer*
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>>
>>>> Unmesha Sreeveni U.B
>>>>
>>>> *Junior Developer*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>>
>>> *Junior Developer*
>>>
>>>
>>>
>>
>
--
*Thanks & Regards*

Unmesha Sreeveni U.B

*Junior Developer*