Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Java Heap space Exception


Copy link to this message
-
Re: Java Heap space Exception
Thx Rob Blah
On Fri, Nov 15, 2013 at 6:54 PM, Rob Blah <[EMAIL PROTECTED]> wrote:

> MapReduce
> http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
> "We’ll thus assign 4 GB for Map task Containers, and 8 GB for Reduce tasks
> Containers."
>
> You can try editing configuration file:
> <name>mapreduce.map.memory.mb</name>
> <value>?</value>
>
> <name>mapreduce.reduce.memory.mb</name>
>  <value>?</value>
>
> Or set these in a driver.
>
> If you have custom AM, try this:
> Driver:
> capability.setMemory(amMemory);
> appContext.setResource(capability);
>
> AM:
> Resource capability = Records.newRecord(Resource.class);
> capability.setMemory(containerMemory);
> AMRMClient.ContainerRequest request = new
> AMRMClient.ContainerRequest(capability, null, null, pri);
> OR AM (use java command to set heap size):
> vargs.add("-Xmx" + containerMemory + "m");
>
>
>
>
>
>
> 2013/11/15 Rob Blah <[EMAIL PROTECTED]>
>
>> Question have you tried giving more memory to the containers?
>>
>>
>> 2013/11/15 unmesha sreeveni <[EMAIL PROTECTED]>
>>
>>> ys found
>>> tried -D mapred.child.java.opts=-Xmx4096M on the command line:
>>>
>>>
>>> On Fri, Nov 15, 2013 at 3:01 PM, unmesha sreeveni <[EMAIL PROTECTED]
>>> > wrote:
>>>
>>>> hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata /user/unmesha
>>>> /out
>>>>
>>>>
>>>>
>>>> On Fri, Nov 15, 2013 at 2:53 PM, unmesha sreeveni <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> When i tried to execute my program with 100 MB file i am getting
>>>>> JavaHeapSpace Exception
>>>>>
>>>>>
>>>>> >hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata
>>>>> /user/inputdata/out
>>>>>
>>>>> How to increase the heap size through commandline
>>>>>
>>>>>
>>>>> --
>>>>> *Thanks & Regards*
>>>>>
>>>>> Unmesha Sreeveni U.B
>>>>>
>>>>> *Junior Developer*
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>>
>>>> Unmesha Sreeveni U.B
>>>>
>>>> *Junior Developer*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>>
>>> *Junior Developer*
>>>
>>>
>>>
>>
>
--
*Thanks & Regards*

Unmesha Sreeveni U.B

*Junior Developer*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB