Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> error running pi program


Copy link to this message
-
Re: error running pi program
On 10 November 2012 00:48, yinghua hu <[EMAIL PROTECTED]> wrote:

> Hi, Ted and Andy
>
> I tried both internal and external hostnames. They both worked. But I will
> elect external host name since this way I can browsing the hdfs file
> structure and other information from a computer not in the cluster.
>
> Thanks!
>

I wouldn't recommend this:

If you hook up your DNs to the NN via the external addresses, Amazon will
bill you for external traffic, which could be an unpleasant surprise at the
end of the month.

If you deploy using Apache Whirr then it gives you a command to run an SSH
proxy.

I've filed a JIRA for the webhdfs feature in Hadoop 1.0.3+ -which offers a
way to get at the filesystem via HTTP to not have the NN issue 302
redirects to the specific DN, but instead act as a proxy; there is a
cluster proxy service (with the same API) that can do this, but for small
EC2 clusters using the NN directly would be easier. What I haven't done is
implemented it -if anyone else wants to I'll gladly review the code (and
its tests)
-Steve
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB