Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> getAllocatedContainers() is not returning when ran against 2.0.3-alpha

Krishna Kishore Bonagiri 2013-04-03, 15:05
Copy link to this message
Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
If I understand your question, you are expecting all the containers to be allocated in one go? Or are you seeing your application hang because it asked for 10 containers but it only received a total of 9 even after repeated calls to the RM?  

There is no guarantee that you will be allocated all your requested containers in a single allocate() call. The application is expected to keep on making allocate calls to the RM at regular intervals and the RM will give it any newly allocated containers in the period  since the last allocate call.

The AMRMClient is  a library that should make it easier for someone to write an application master. Yes - it is the recommended approach. Please file jiras if you see use-cases that are not addressed by it when implementing your AM.

-- Hitesh

On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:

> Hi,
>  I am running an application that had been written and working well with the hadoop-2.0.0-alpha but when I am running the same against 2.0.3-alpha, the getAllocatedContainers() method called on AMResponse is not returning all the containers allocated sometimes. For example, I request for 10 containers and this method gives me only 9 containers sometimes, and when I looked at the log of Resource Manager, the 10th container is also allocated. It happens only sometimes randomly and works fine all other times. If I send one more request for the remaining container to RM, it could allocate that container. I am running only one application at a time, but 1000s of them one after another.
>   Is this problem seen by anyone else? I have seen that there are significant number of changes in the way ApplicationMaster.java is written now for the Distributed Shell example. There is now AMRMClient instead of AMRMProtocol. There is method addContainerRequest() which is not there earlier, and there is ConatainerRequest class now which was ResourceRequest earlier, etc.  Is it adivsable to re-write my application in a similar way?
> Thanks,
> Kishore