Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> a question on NameNode


Copy link to this message
-
Re: a question on NameNode
Hi,

Am 19.11.2012 um 15:27 schrieb "Kartashov, Andy" <[EMAIL PROTECTED]>:

> I am learning that NN doesn’t persistently store block locations. Only file names and heir permissions as well as file blocks. It is said that locations come from DataNodes when NN starts.
>  
> So, how does it work?
>  
> Say we only have one file A.txt in our HDFS that is split into 4 blocks 1,2,3,4 (no replication), with block 1-2 residing on DN1 and blocks 3,4 on DN2.
>  
> When we start NN it reads it metastore and tries to locate and map the locations of 4 blocks of file A.txt??

when a NameNode starts, it does that in safe mode. Like you said, it doesn't know where the blocks are. The DataNodes send a list of all of their local block IDs (so called block reports). Once the NameNode knows about the locations of most blocks (99,9%, configurable number), it will leave safe mode and HDFS is back.

Kai

--
Kai Voigt
[EMAIL PROTECTED]
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB