Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Issues with ResourceManager scheduling functions


Copy link to this message
-
Re: Issues with ResourceManager scheduling functions
Are you requesting the different container types at different priorities?

-Sandy
On Sun, Dec 8, 2013 at 8:52 PM, Bill Q <[EMAIL PROTECTED]> wrote:

> Hi,
> I recently implemented an application using YARN. Everything works fine,
> except for the illogical scheduling behavior of the ResourceManger, which
> might be caused by the heterogeneity of the cluster.
>
> I have two types of computing node in the cluster. 10 nodes with 128G
> memory and 10 nodes with 64G memory. In the application, I create two types
> of container. Type A container would ask for 50G of memory and Type B
> container would ask for 100G of memory.
>
> The problem is that if I created 50 Type B containers (100G), and 50 Type
> A containers (50G), Only Type B container (100G) are running. Instead of
> running the Type A container (50G), all the nodes with 64G memory are
> idling. And only the nodes with 128G memory are running Type B container
> (100G). This doesn't make too much sense to me.
>
> If I only created Type A containers (50G), all the nodes would be able run
> the containers. The 128G node would run two containers simultaneously. So,
> the problem seems to be connected to the different types of containers.
>
> Does anybody have any suggestions to fix this issue?
>
> Many thanks.
>
>
> Bill
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB