Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # general >> Help: viewing Hadoop web pages via URL to a back-end network, and specifying an alt config directory


Copy link to this message
-
Help: viewing Hadoop web pages via URL to a back-end network, and specifying an alt config directory
     I have two questions...

      First, our Hadoop cluster uses a back-end, non-routed network for
transfering data, it is not routed and not accessible beyond the
cluster.  Thus, the resulting URLs the Hadoop web pages display are
inaccessible.

      While looking around, I found a similar, older thread about this at:

http://www.mail-archive.com/[EMAIL PROTECTED]/msg10664.html

     Our developers need to access the URLs to the various job info
directly -- but I'm not sure how I can do this.   I don't think setting
dfs.datanode.dns.interface will have any affect.

     I could also NAT a port range from the front-end if I needed to,
with iptables... but at this point, this is seems cosmetic. Surely
someone else has run into this :-)  Here's an example of what I'm seeing
displayed on our hadoop pages:

      The Hadoop web page displays this text:

hadoop.our-backend.com.:hadoop.our-frontend.com/127.0.0.1:42302

      with the actual URL (click) being:

http://hadoop.our-backend.com.:50060/

Note the added "." in the actual links, which is not coming from our
*.xml config files or /etc/hosts.

The display shows backend -> frontend -> localhost.  Seems a little
convoluted; at least, I know our developers are getting irritated by it :-)

     There must be a relatively simple solution to this - as surely
others have a similar configuration?

      My second question is partly due to some confusion on my part from
reading the documentation.   We want to manage our Hadoop installation
for each system by moving out the /conf directory to another location.  
There appear to be a few places you can do this, including the command
line.   What I can't seem to figure out is where (preferrably at
startup) can I place this value so that it's /inherited/ across the
board, so I don't have to worry about multiple edits?
Thanks!