Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Basic Doubt in Hadoop


Copy link to this message
-
Re: Basic Doubt in Hadoop

You are correct, map outputs are stored in LFS not in HDFS.

Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: Ramesh R Nair <[EMAIL PROTECTED]>
Date: Wed, 17 Apr 2013 13:06:32
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Subject: Re: Basic Doubt in Hadoop

Hi Bejoy,

   Regarding the output of Map phase, does  Hadoop store it in local fs or
in HDFS.
   I believe it is in the former. Correct me if I am wrong.

Regards
Ramesh
On Wed, Apr 17, 2013 at 10:30 AM, <[EMAIL PROTECTED]> wrote:

> The data is in HDFS in case of WordCount MR sample.
>
> In hdfs, you have the metadata in NameNode and actual data as blocks
> replicated across DataNodes.
>
> In case of reducer, If a reducer is running on a particular node then you
> have one replica of the blocks in the same node (If there is no space
> issues) and rest replicas on other nodes.
> Regards
> Bejoy KS
>
> Sent from remote device, Please excuse typos
> ------------------------------
> *From: * Raj Hadoop <[EMAIL PROTECTED]>
> *Date: *Tue, 16 Apr 2013 21:49:34 -0700 (PDT)
> *To: *[EMAIL PROTECTED]<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Basic Doubt in Hadoop
>
> Hi,
>
> I am new to Hadoop. I started reading the standard Wordcount program. I
> got this basic doubt in Hadoop.
>
> After the Map - Reduce is done, where is the output generated?  Does the
> reducer ouput sit on individual DataNodes ? Please advise.
>
>
> Thanks,
> Raj
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB