Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - How to change topology


Copy link to this message
-
Re: How to change topology
Steve Loughran 2012-10-09, 19:17
On 9 October 2012 16:51, Shinichi Yamashita <[EMAIL PROTECTED]> wrote:

> Hi Steve,
>
> Thank you for your reply.
>
>
> > no, it's the Namenode and JobTracker that needs to be restarted;
> > they are the bits that care where the boxes are.
>
> I confirmed it in my cluster, and I understood it as follows.
> First, the resolved node information is recorded in ConcurrentHashMap.
> Next same node toplogy decision, HashMap data is given priority to over a
> script.
> The HashMap data become invalid by restart of NameNode and JobTracker.
>

That's right. There's been lots of discussion on adding live topology
resetting.

There's also a large unfinished JIRA in my name to do better topology work
in Hadoop 2,
https://issues.apache.org/jira/browse/HADOOP-8231

Including the ability to do local checks of a topology script
https://issues.apache.org/jira/browse/HADOOP-8232

Supporting topology dump and reset from any of the manager nodes would come
after.

-steve