Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> DefaultResourceCalculator class not found, ResourceManager fails to start.


Copy link to this message
-
Re: DefaultResourceCalculator class not found, ResourceManager fails to start.
Problem solved. Thank you for your help.

@Ted Yu
Other issues where my mistakes. I have a dedicated script which
updates/builds/"deploys" YARN from sources. I was starting NN with the
"-upgrade" option which unsynchronized NN version, also leading to broken
DN. Quick NN format and deletion of DN data solved the issue (I am working
on a sandbox cluster, so that is not a problem). I have modified the script
to start the NN without the upgrade option.

Two quick questions:
- when should I use the NN upgrade option, should it be only used to
upgrade NN between new version (example 2.0.4 -> 2.0.5). How can I automate
this process?
- Is the "design/functionality" of my magical script correct? How can I
avoid future problems like the solved one?

YARN update script
- update src to trunk (opt)
- package YARN
- build dist (tar ball)
- unpack new_dist
- overwrite new_dist conf with prev_dist conf (this has lead to the problem
with DefaultResourceCalculator, my conf is bare minimum to work in
pseudo-distributed mode)
- start YARN

For any suggestions I would be grateful.

regards
tmp

2013/8/12 Ted Yu <[EMAIL PROTECTED]>

> Can you check the config entry
> for yarn.scheduler.capacity.resource-calculator ?
> It should point
> to org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
>
> bq. I was able to fix all issues
>
> What other issues came up ?
>
> Thanks
>
>
> On Sun, Aug 11, 2013 at 2:07 PM, Rob Blah <[EMAIL PROTECTED]> wrote:
>
>> Hi again
>>
>> From a little investigation I have performed I have observed the
>> following. I assume the module responsible for this class is
>> hadoop-yarn-common.
>>
>> During RM init it crashes since it is looking for a class
>> DefaultResourceCalculator in
>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator,
>> while the class is present in hadoop-yarn-common-3.0.0-SNAPSHOT.jar but
>> under org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator. Thus
>> the RM crashes. Anybody knows how I can fix this? I would very grateful for
>> any help.
>>
>> regards
>> tmp
>>
>>
>> 2013/8/11 Rob Blah <[EMAIL PROTECTED]>
>>
>>> Hi
>>>
>>> I have a strange problem, regarding missing class, the
>>> DefaultResourceCalculator. I have a single node sandbox cluster working in
>>> a pseudo-distributed mode. The cluster was working fine yesterday, however
>>> today it stopped working. I was able to fix all issues except the following
>>> problem in ResourceManager:
>>> 2013-08-11 12:12:42,425 FATAL
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error
>>> starting ResourceManager
>>> java.lang.RuntimeException: java.lang.RuntimeException:
>>> java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>>     at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1753)
>>>     at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333)
>>>     at
>>> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:258)
>>>     at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:241)
>>>     at
>>> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
>>>     at
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:826)
>>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator
>>> not found
>>>     at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1721)
>>>     at
>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1745)
>>>     ... 5 more
>>> Caused by: java.lang.ClassNotFoundException: Class
>>> org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator