Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Issues with ResourceManager scheduling functions

Copy link to this message
Re: Issues with ResourceManager scheduling functions
Are you requesting the different container types at different priorities?

On Sun, Dec 8, 2013 at 8:52 PM, Bill Q <[EMAIL PROTECTED]> wrote:

> Hi,
> I recently implemented an application using YARN. Everything works fine,
> except for the illogical scheduling behavior of the ResourceManger, which
> might be caused by the heterogeneity of the cluster.
> I have two types of computing node in the cluster. 10 nodes with 128G
> memory and 10 nodes with 64G memory. In the application, I create two types
> of container. Type A container would ask for 50G of memory and Type B
> container would ask for 100G of memory.
> The problem is that if I created 50 Type B containers (100G), and 50 Type
> A containers (50G), Only Type B container (100G) are running. Instead of
> running the Type A container (50G), all the nodes with 64G memory are
> idling. And only the nodes with 128G memory are running Type B container
> (100G). This doesn't make too much sense to me.
> If I only created Type A containers (50G), all the nodes would be able run
> the containers. The 128G node would run two containers simultaneously. So,
> the problem seems to be connected to the different types of containers.
> Does anybody have any suggestions to fix this issue?
> Many thanks.
> Bill