Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> hadoop imbalanced topology


Copy link to this message
-
hadoop imbalanced topology
Hi,

Is it right that hadoop Network Topology design treat an imbalanced
topology as an invalid topology?

This is mentioned here
https://issues.apache.org/jira/browse/MAPREDUCE-4095(TestJobInProgress#testLocalityuses
a bogus topology.)
said:"(
https://issues.apache.org/jira/secure/attachment/12345251/Rack_aware_HDFS_proposal.pdf)
which does not support imbalanced topology. "

But when I re-read the desigh pdf, there just draw a balanced topology but
really not mentioned it will not support imbalanced topology.

Also I found this JIRA:
https://issues.apache.org/jira/browse/HADOOP-8159(NetworkTopology: getLeaf
should check for invalid topologies).

Just no why for my question, because of the design(Right?):
*Why below is a invalid topology(except imbalanced), for hadoop rack
awareness:*
*serverA /default-rack    ##default rack*
*serverB /dc1/rack2    ##actually the node has a pre "dc1"*
*Should all of the hadoop datanode have the same level number? If not,
there will be some issues for namenode and jt(like scheduler deadlock)?*

专注于Mysql,MSSQL,Oracle,Hadoop
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB