Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Issues with ResourceManager scheduling functions


Copy link to this message
-
Re: Issues with ResourceManager scheduling functions
Sandy Ryza 2013-12-09, 07:53
Are you requesting the different container types at different priorities?

-Sandy
On Sun, Dec 8, 2013 at 8:52 PM, Bill Q <[EMAIL PROTECTED]> wrote:

> Hi,
> I recently implemented an application using YARN. Everything works fine,
> except for the illogical scheduling behavior of the ResourceManger, which
> might be caused by the heterogeneity of the cluster.
>
> I have two types of computing node in the cluster. 10 nodes with 128G
> memory and 10 nodes with 64G memory. In the application, I create two types
> of container. Type A container would ask for 50G of memory and Type B
> container would ask for 100G of memory.
>
> The problem is that if I created 50 Type B containers (100G), and 50 Type
> A containers (50G), Only Type B container (100G) are running. Instead of
> running the Type A container (50G), all the nodes with 64G memory are
> idling. And only the nodes with 128G memory are running Type B container
> (100G). This doesn't make too much sense to me.
>
> If I only created Type A containers (50G), all the nodes would be able run
> the containers. The 128G node would run two containers simultaneously. So,
> the problem seems to be connected to the different types of containers.
>
> Does anybody have any suggestions to fix this issue?
>
> Many thanks.
>
>
> Bill
>