Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Basic Doubt in Hadoop


Copy link to this message
-
Re: Basic Doubt in Hadoop
@Bejoy
Adding a little bit here,the ouput of map task writes first to a memory
buffer, and when contents reaches a threshold a background thread will
write the contents to disk.

Niranjan Singh
On Wed, Apr 17, 2013 at 1:06 PM, Ramesh R Nair <[EMAIL PROTECTED]>wrote:

> Hi Bejoy,
>
>    Regarding the output of Map phase, does  Hadoop store it in local fs or
> in HDFS.
>    I believe it is in the former. Correct me if I am wrong.
>
> Regards
> Ramesh
>
>
> On Wed, Apr 17, 2013 at 10:30 AM, <[EMAIL PROTECTED]> wrote:
>
>> The data is in HDFS in case of WordCount MR sample.
>>
>> In hdfs, you have the metadata in NameNode and actual data as blocks
>> replicated across DataNodes.
>>
>> In case of reducer, If a reducer is running on a particular node then you
>> have one replica of the blocks in the same node (If there is no space
>> issues) and rest replicas on other nodes.
>> Regards
>> Bejoy KS
>>
>> Sent from remote device, Please excuse typos
>> ------------------------------
>> *From: * Raj Hadoop <[EMAIL PROTECTED]>
>> *Date: *Tue, 16 Apr 2013 21:49:34 -0700 (PDT)
>> *To: *[EMAIL PROTECTED]<[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Basic Doubt in Hadoop
>>
>> Hi,
>>
>> I am new to Hadoop. I started reading the standard Wordcount program. I
>> got this basic doubt in Hadoop.
>>
>> After the Map - Reduce is done, where is the output generated?  Does the
>> reducer ouput sit on individual DataNodes ? Please advise.
>>
>>
>> Thanks,
>> Raj
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB