Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Use VIP for DataNodes


Copy link to this message
-
Re: Use VIP for DataNodes

FWIW, Y! and LI (and I'm sure others) have been using failover A records for years.  The only known problem/limitation is exactly what Dhruba states.

On Jan 29, 2011, at 11:19 PM, Dhruba Borthakur wrote:

> If you want to protect urself against a rack-switch failure, then you would
> want to keep ur primary and standby on machines in different racks.
> Typically, machines in different racks have different subnets.
>
> thanks
> dhruba
>
> On Wed, Jan 26, 2011 at 7:10 AM, mac fang <[EMAIL PROTECTED]> wrote:
>
>> Hi, dhruba,
>>
>> Thanks for sharing. Seems we only need to let NameNode & Backup Node in the
>> same subnet. Why it is a main problem?
>>
>> regards
>> macf
>>
>> On Wed, Jan 26, 2011 at 4:14 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>>> Thank you for correcting that :)
>>>
>>> On Wed, Jan 26, 2011 at 11:17 AM, Dhruba Borthakur <[EMAIL PROTECTED]>
>>> wrote:
>>>> Fb does not use the VIP approach, we tried that but quickly found out
>>> some
>>>> limitations, one main problem being that the failover server pair has
>> to
>>> be
>>>> in the same subnet (for VIP to work). Instead we now use the AvatarNode
>>>> integrated with Zookeeper.
>>>>
>>>> -dhruba
>>>>
>>>> On Tue, Jan 25, 2011 at 6:12 PM, Harsh J <[EMAIL PROTECTED]>
>> wrote:
>>>>
>>>>> This is a good approach, is used by many to mask the NameNode address
>>>>> for the DataNodes; and is also good to use while using BackupNode
>>>>> (Even Facebook does this).
>>>>>
>>>>> On Wed, Jan 26, 2011 at 7:38 AM, mac fang <[EMAIL PROTECTED]>
>> wrote:
>>>>>> Hi, guys,
>>>>>>
>>>>>> Do we have any plan to enable VIP between NameNode and DataNodes, I
>>> mean
>>>>>> multiple NameNodes and we use a VIP stands before the NNs. Then DNs
>>> only
>>>>>> need to connect to the VIP. I don't know if it is valuable we have
>>> this
>>>>> impl
>>>>>> in our hdsf and if there has any issues?
>>>>>>
>>>>>> Any hints are valuable for me. :)
>>>>>>
>>>>>> regards
>>>>>> macf
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Harsh J
>>>>> www.harshj.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Connect to me at http://www.facebook.com/dhruba
>>>>
>>>
>>>
>>>
>>> --
>>> Harsh J
>>> www.harshj.com
>>>
>>
>
>
>
> --
> Connect to me at http://www.facebook.com/dhruba