Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Hadoop 2.2.0 YARN isolation and Windows


Copy link to this message
-
Re: Hadoop 2.2.0 YARN isolation and Windows
Hi Kevin,

*> 2. According to
http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html
<http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html>,
Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for
secure container isolation in Windows? It seems that the ContainerExecutor
and DefaultContainerExecutor can detect and run on Windows, but the secure
LinuxContainerExecutor are for *nix systems, so is there anything in place
for maximum security like LCE is?*

Cluster security is not supported on Windows at the moment.
*> 1. Does YARN need to run on the same machines that are hosting the HDFS
services or can HDFS be remote of a YARN cluster? If this done by placing
the remote HDFS cluster's configuration files (core-site.xml and
hdfs-site.xml) on the YARN cluster's machines?*

*> 3. If 1 is yes, then is it possible to have a cluster mixed with both
Linux and Windows machines running YARN and working together?*
It should work in theory if you get the configuration right - I have not
tried it out so I am not sure. YARN containers and HDFS datanodes should be
collocated for good performance. The MapReduce compute model especially
depends on access to fast local storage.

On Thu, Jan 9, 2014 at 1:19 PM, Kevin <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Three questions about the new Hadoop release regarding YARN:
>
> 1. Does YARN need to run on the same machines that are hosting the HDFS
> services or can HDFS be remote of a YARN cluster? If this done by placing
> the remote HDFS cluster's configuration files (core-site.xml and
> hdfs-site.xml) on the YARN cluster's machines?
>
> 2. According to
> http://www.i-programmer.info/news/197-data-mining/6518-hadoop-2-introduces-yarn.html,
> Hadoop 2.2.0 supports Microsoft Windows. How do/Can you configure YARN for
> secure container isolation in Windows? It seems that the ContainerExecutor
> and DefaultContainerExecutor can detect and run on Windows, but the secure
> LinuxContainerExecutor are for *nix systems, so is there anything in place
> for maximum security like LCE is?
>
> 3. If 1 is yes, then is it possible to have a cluster mixed with both
> Linux and Windows machines running YARN and working together?
>
> Thanks,
> Kevin
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB