Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> HDFS - MapReduce coupling


+
Matthew John 2011-05-02, 06:48
Copy link to this message
-
Re: HDFS - MapReduce coupling
Yes.  There is quite a bit of need for the local file system in clustered
mode.

For one think, all of the shuffle intermediate files are on local disk.  For
another, the distributed cache is actually stored on local disk.

HFDS is a frail vessel that cannot cope with all the needs.

On Sun, May 1, 2011 at 11:48 PM, Matthew John <[EMAIL PROTECTED]>wrote:

> ...
> 2) Does the Hadoop system utilize the local storage directly for any
> purpose
> (without going through the HDFS) in clustered mode?
>
>
+
Matthew John 2011-05-02, 07:16
+
Matthew John 2011-05-02, 11:23
+
James Seigel 2011-05-02, 11:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB