-Re: Basic Doubt in Hadoop
maisnam ns 2013-04-17, 09:06
Adding a little bit here,the ouput of map task writes first to a memory
buffer, and when contents reaches a threshold a background thread will
write the contents to disk.
On Wed, Apr 17, 2013 at 1:06 PM, Ramesh R Nair <[EMAIL PROTECTED]>wrote:
> Hi Bejoy,
> Regarding the output of Map phase, does Hadoop store it in local fs or
> in HDFS.
> I believe it is in the former. Correct me if I am wrong.
> On Wed, Apr 17, 2013 at 10:30 AM, <[EMAIL PROTECTED]> wrote:
>> The data is in HDFS in case of WordCount MR sample.
>> In hdfs, you have the metadata in NameNode and actual data as blocks
>> replicated across DataNodes.
>> In case of reducer, If a reducer is running on a particular node then you
>> have one replica of the blocks in the same node (If there is no space
>> issues) and rest replicas on other nodes.
>> Bejoy KS
>> Sent from remote device, Please excuse typos
>> *From: * Raj Hadoop <[EMAIL PROTECTED]>
>> *Date: *Tue, 16 Apr 2013 21:49:34 -0700 (PDT)
>> *To: *[EMAIL PROTECTED]<[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Basic Doubt in Hadoop
>> I am new to Hadoop. I started reading the standard Wordcount program. I
>> got this basic doubt in Hadoop.
>> After the Map - Reduce is done, where is the output generated? Does the
>> reducer ouput sit on individual DataNodes ? Please advise.