Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> DefaultResourceCalculator class not found, ResourceManager fails to start.


Copy link to this message
-
Re: DefaultResourceCalculator class not found, ResourceManager fails to start.
Problem solved. Thank you for your help.

@Ted Yu
Other issues where my mistakes. I have a dedicated script which
updates/builds/"deploys" YARN from sources. I was starting NN with the
"-upgrade" option which unsynchronized NN version, also leading to broken
DN. Quick NN format and deletion of DN data solved the issue (I am working
on a sandbox cluster, so that is not a problem). I have modified the script
to start the NN without the upgrade option.

Two quick questions:
- when should I use the NN upgrade option, should it be only used to
upgrade NN between new version (example 2.0.4 -> 2.0.5). How can I automate
this process?
- Is the "design/functionality" of my magical script correct? How can I
avoid future problems like the solved one?

YARN update script
- update src to trunk (opt)
- package YARN
- build dist (tar ball)
- unpack new_dist
- overwrite new_dist conf with prev_dist conf (this has lead to the problem
with DefaultResourceCalculator, my conf is bare minimum to work in
pseudo-distributed mode)
- start YARN

For any suggestions I would be grateful.

regards
tmp

2013/8/12 Ted Yu <[EMAIL PROTECTED]>

> Can you check the config entry
> for yarn.scheduler.capacity.resource-calculator ?
> It should point
> to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
>
> bq. I was able to fix all issues
>
> What other issues came up ?
>
> Thanks
>
>
> On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <[EMAIL PROTECTED]> wrote:
>
>> Hi again
>>
>> From a little investigation I have performed I have observed the
>> following. I assume the module responsible for this class is
>> hadoop-yarn-common.
>>
>> During RM init it crashes since it is looking for a class
>> DefaultResourceCalculator in
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
>> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
>> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
>> the RM crashes. Anybody knows how I can fix this? I would very grateful for
>> any help.
>>
>> regards
>> tmp
>>
>>
>> 2013/8/11 Rob Blah <[EMAIL PROTECTED]>
>>
>>> Hi
>>>
>>> I have a strange problem, regarding missing class, the
>>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>>> today it stopped working. I was able to fix all issues except the following
>>> problem in ResourceManager:
>>> 2013-08-11 12:12:42,425 FATAL
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>>> starting ResourceManager
>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>> java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>>     at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>>>     at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>>>     at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>>>     at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>>>     at
>>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>>     at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>>     at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>>>     at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>>>     ... 5 more
>>> Caused by: java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB