Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # dev - Merging Namenode Federation feature (HDFS-1052) to trunk


+
Suresh Srinivas 2011-03-03, 22:41
+
Allen Wittenauer 2011-03-12, 16:43
+
Konstantin Shvachko 2011-03-14, 17:28
+
Dhruba Borthakur 2011-03-14, 17:43
+
Konstantin Shvachko 2011-03-15, 01:12
+
Travis Crawford 2011-03-15, 05:36
+
suresh srinivas 2011-03-15, 06:19
+
Konstantin Shvachko 2011-03-16, 22:52
+
suresh srinivas 2011-03-16, 23:54
+
suresh srinivas 2011-03-16, 23:55
Copy link to this message
-
Re: Merging Namenode Federation feature (HDFS-1052) to trunk
Sanjay Radia 2011-03-14, 17:57

On Mar 12, 2011, at 8:43 AM, Allen Wittenauer wrote:

>
> On Mar 3, 2011, at 2:41 PM, Suresh Srinivas wrote:
>
>> We have started pushing changes for namenode federation in to the  
>> feature branch HDFS-1052. The work items are created as subtask of  
>> the jira HDFS-1052 and are based on the design document published  
>> in the same jira. By the end of this week, we will complete pushing  
>> the changes to HDFS-1052 branch. Though the changes in these jiras  
>> are already committed, please do provide your feedback on either  
>> HDFS-1052 or its subtasks. New items that come out of the feedback  
>> will be addressed in new jiras.
>
>>
>> Current status of the development:
>> # The testing of this feature is underway. Most of the basic  
>> functionality has been tested both for a single namenode cluster  
>> (for backward compatibility) and with multiple namenodes.
>> # All the existing tests and newly added tests pass (same as trunk).
>>
>> We plan on merging this branch to trunk after a week or two. This  
>> will help us continue make future changes on the trunk. I will send  
>> an announcement before merging the federation branch into trunk.
>>
>
> It sounds like merging into trunk is extremely premature.  That  
> said, I'm still trying to understand the why's around this.
>
> To me, this series of changes looks like it is going to make  
> running a grid much much harder for very little benefit.  In  
> particular, I don't see the difference between running multiple NN/
> DN combinations verses running federation, especially with client  
> side mount tables in play.

Main difference between independent HDFS clusters and HDFS federation  
is that in federation one can shares the storage of the DNs and the DNs.
There is a very detailed document that describes this on the Jira.

If you are running a single NN and you don't need the scaling then  
running and managing hadoop is for all practical purposes unchanged.
sanjay
>
+
Sanjay Radia 2011-03-21, 23:08
+
Brian Bockelman 2011-03-21, 23:25
+
suresh srinivas 2011-03-24, 09:28
+
Allen Wittenauer 2011-03-22, 20:11
+
suresh srinivas 2011-03-24, 09:34