Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> Use VIP for DataNodes


+
mac fang 2011-01-26, 02:08
+
Harsh J 2011-01-26, 02:12
+
mac fang 2011-01-26, 03:14
+
Dhruba Borthakur 2011-01-26, 05:47
+
Harsh J 2011-01-26, 08:14
+
mac fang 2011-01-26, 15:10
+
Dhruba Borthakur 2011-01-30, 07:19
Copy link to this message
-
Re: Use VIP for DataNodes

FWIW, Y! and LI (and I'm sure others) have been using failover A records for years.  The only known problem/limitation is exactly what Dhruba states.

On Jan 29, 2011, at 11:19 PM, Dhruba Borthakur wrote:

> If you want to protect urself against a rack-switch failure, then you would
> want to keep ur primary and standby on machines in different racks.
> Typically, machines in different racks have different subnets.
>
> thanks
> dhruba
>
> On Wed, Jan 26, 2011 at 7:10 AM, mac fang <[EMAIL PROTECTED]> wrote:
>
>> Hi, dhruba,
>>
>> Thanks for sharing. Seems we only need to let NameNode & Backup Node in the
>> same subnet. Why it is a main problem?
>>
>> regards
>> macf
>>
>> On Wed, Jan 26, 2011 at 4:14 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>>> Thank you for correcting that :)
>>>
>>> On Wed, Jan 26, 2011 at 11:17 AM, Dhruba Borthakur <[EMAIL PROTECTED]>
>>> wrote:
>>>> Fb does not use the VIP approach, we tried that but quickly found out
>>> some
>>>> limitations, one main problem being that the failover server pair has
>> to
>>> be
>>>> in the same subnet (for VIP to work). Instead we now use the AvatarNode
>>>> integrated with Zookeeper.
>>>>
>>>> -dhruba
>>>>
>>>> On Tue, Jan 25, 2011 at 6:12 PM, Harsh J <[EMAIL PROTECTED]>
>> wrote:
>>>>
>>>>> This is a good approach, is used by many to mask the NameNode address
>>>>> for the DataNodes; and is also good to use while using BackupNode
>>>>> (Even Facebook does this).
>>>>>
>>>>> On Wed, Jan 26, 2011 at 7:38 AM, mac fang <[EMAIL PROTECTED]>
>> wrote:
>>>>>> Hi, guys,
>>>>>>
>>>>>> Do we have any plan to enable VIP between NameNode and DataNodes, I
>>> mean
>>>>>> multiple NameNodes and we use a VIP stands before the NNs. Then DNs
>>> only
>>>>>> need to connect to the VIP. I don't know if it is valuable we have
>>> this
>>>>> impl
>>>>>> in our hdsf and if there has any issues?
>>>>>>
>>>>>> Any hints are valuable for me. :)
>>>>>>
>>>>>> regards
>>>>>> macf
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Harsh J
>>>>> www.harshj.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Connect to me at http://www.facebook.com/dhruba
>>>>
>>>
>>>
>>>
>>> --
>>> Harsh J
>>> www.harshj.com
>>>
>>
>
>
>
> --
> Connect to me at http://www.facebook.com/dhruba
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB