Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Re: Basic Doubt in Hadoop


+
bejoy.hadoop@... 2013-04-17, 05:00
Copy link to this message
-
Re: Basic Doubt in Hadoop
maisnam ns 2013-04-17, 09:06
@Bejoy
Adding a little bit here,the ouput of map task writes first to a memory
buffer, and when contents reaches a threshold a background thread will
write the contents to disk.

Niranjan Singh
On Wed, Apr 17, 2013 at 1:06 PM, Ramesh R Nair <[EMAIL PROTECTED]>wrote:

> Hi Bejoy,
>
>    Regarding the output of Map phase, does  Hadoop store it in local fs or
> in HDFS.
>    I believe it is in the former. Correct me if I am wrong.
>
> Regards
> Ramesh
>
>
> On Wed, Apr 17, 2013 at 10:30 AM, <[EMAIL PROTECTED]> wrote:
>
>> The data is in HDFS in case of WordCount MR sample.
>>
>> In hdfs, you have the metadata in NameNode and actual data as blocks
>> replicated across DataNodes.
>>
>> In case of reducer, If a reducer is running on a particular node then you
>> have one replica of the blocks in the same node (If there is no space
>> issues) and rest replicas on other nodes.
>> Regards
>> Bejoy KS
>>
>> Sent from remote device, Please excuse typos
>> ------------------------------
>> *From: * Raj Hadoop <[EMAIL PROTECTED]>
>> *Date: *Tue, 16 Apr 2013 21:49:34 -0700 (PDT)
>> *To: *[EMAIL PROTECTED]<[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Basic Doubt in Hadoop
>>
>> Hi,
>>
>> I am new to Hadoop. I started reading the standard Wordcount program. I
>> got this basic doubt in Hadoop.
>>
>> After the Map - Reduce is done, where is the output generated?  Does the
>> reducer ouput sit on individual DataNodes ? Please advise.
>>
>>
>> Thanks,
>> Raj
>>
>
>