Hi,

Am trying to bring high availability in namenode(in hadoop cluster) for
small setup.

So the idea is to keep one extra namenode and use terracotta to replicate
the log(in memory details-jvm) from primary datacode to standby namenode, if
primary goes down secondary can take the primary role.

Now am bit confused how to start second namenode while the primary name node
up and running(with datanode connected) with keep relying the info from
primary thro terracotta. Is there a way to achieve this without restarting
namenode or datanode, any patch?

Please help and suggest me.

Regards,
Gokul

Gokul,

Terracotta is a fascinating framework which might be a bit of an overshot for a
small setup as well as adding another level of complexity in the form
Terracotta framework.

Quite interesting approach for NN's HA is Jini (aka ASF River) along with
JavaSpaces for the distributed storage of the meta-data. Using Jini also gives
you for free such benefits as auto-discovering making hot-swap of NN a non-issue.

I've been musing over this idea for some time and am trying to find some time
to actually experiment with it.

Cos

On Fri, Sep 23, 2011 at 11:36PM, Gokul Prasad wrote:
>    Hi,
>    Am trying to bring high availability in namenode(in hadoop cluster) for
>    small setup.
>    So the idea is to keep one extra namenode and use terracotta to replicate
>    the log(in memory details-jvm) from primary datacode to standby namenode,
>    if primary goes down secondary can take the primary role.
>    Now am bit confused how to start second namenode while the primary name
>    node up and running(with datanode connected) with keep relying the info
>    from primary thro terracotta. Is there a way to achieve this without
>    restarting namenode or datanode, any patch?
>    Please help and suggest me.
>    Regards,
>    Gokul

Cos,

Am planning to implement this idea in bigger cluster but first i want
to get confidence in small cluster. I do read about terracotta and
inspired by this idea.

-gokul

On Fri, Sep 23, 2011 at 11:57 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
> Gokul,
>
> Terracotta is a fascinating framework which might be a bit of an overshot for a
> small setup as well as adding another level of complexity in the form
> Terracotta framework.
>
> Quite interesting approach for NN's HA is Jini (aka ASF River) along with
> JavaSpaces for the distributed storage of the meta-data. Using Jini also gives
> you for free such benefits as auto-discovering making hot-swap of NN a non-issue.
>
> I've been musing over this idea for some time and am trying to find some time
> to actually experiment with it.
>
> Cos
>
> On Fri, Sep 23, 2011 at 11:36PM, Gokul Prasad wrote:
>>    Hi,
>>    Am trying to bring high availability in namenode(in hadoop cluster) for
>>    small setup.
>>    So the idea is to keep one extra namenode and use terracotta to replicate
>>    the log(in memory details-jvm) from primary datacode to standby namenode,
>>    if primary goes down secondary can take the primary role.
>>    Now am bit confused how to start second namenode while the primary name
>>    node up and running(with datanode connected) with keep relying the info
>>    from primary thro terracotta. Is there a way to achieve this without
>>    restarting namenode or datanode, any patch?
>>    Please help and suggest me.
>>    Regards,
>>    Gokul
>

Is there any reason not to just use something as simple as drbd (which is baked into vanilla linux)? Unless you are already paying terracotta for their software there are plenty of HA setups that are supported through the use of drbd and a virtual ip.

Matt

-----Original Message-----
From: Gokul Prasad [mailto:[EMAIL PROTECTED]]
Sent: Friday, September 23, 2011 2:06 PM
To: [EMAIL PROTECTED]
Subject: Re: hadoop high availability using terracotta

Cos,

Am planning to implement this idea in bigger cluster but first i want
to get confidence in small cluster. I do read about terracotta and
inspired by this idea.

-gokul

On Fri, Sep 23, 2011 at 11:57 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
> Gokul,
>
> Terracotta is a fascinating framework which might be a bit of an overshot for a
> small setup as well as adding another level of complexity in the form
> Terracotta framework.
>
> Quite interesting approach for NN's HA is Jini (aka ASF River) along with
> JavaSpaces for the distributed storage of the meta-data. Using Jini also gives
> you for free such benefits as auto-discovering making hot-swap of NN a non-issue.
>
> I've been musing over this idea for some time and am trying to find some time
> to actually experiment with it.
>
> Cos
>
> On Fri, Sep 23, 2011 at 11:36PM, Gokul Prasad wrote:
>>    Hi,
>>    Am trying to bring high availability in namenode(in hadoop cluster) for
>>    small setup.
>>    So the idea is to keep one extra namenode and use terracotta to replicate
>>    the log(in memory details-jvm) from primary datacode to standby namenode,
>>    if primary goes down secondary can take the primary role.
>>    Now am bit confused how to start second namenode while the primary name
>>    node up and running(with datanode connected) with keep relying the info
>>    from primary thro terracotta. Is there a way to achieve this without
>>    restarting namenode or datanode, any patch?
>>    Please help and suggest me.
>>    Regards,
>>    Gokul
>
This e-mail message may contain privileged and/or confidential information, and is intended to be received only by persons entitled
to receive such information. If you have received this e-mail in error, please notify the sender immediately. Please delete it and
all attachments from any servers, hard drives or any other media. Other use of this e-mail by you is strictly prohibited.

All e-mails and attachments sent and received are subject to monitoring, reading and archival by Monsanto, including its
subsidiaries. The recipient of this e-mail is solely responsible for checking for the presence of "Viruses" or other "Malware".
Monsanto, along with its subsidiaries, accepts no liability for any damage caused by any such code transmitted by or accompanying
this e-mail or any attachment.
The information contained in this email may be subject to the export control laws and regulations of the United States, potentially
including but not limited to the Export Administration Regulations (EAR) and sanctions regulations issued by the U.S. Department of
Treasury, Office of Foreign Asset Controls (OFAC).  As a recipient of this information you are obligated to comply with all
applicable U.S. export laws and regulations.

afaik..drbd approach will take longer to bring standby namenode to
live..am trying to minimise the downtime..like standby namenode will
be relayed with the necessary info thro terracotta and if primary
namenode goes down standby will take its role.

-gokul

On Sat, Sep 24, 2011 at 1:00 AM, GOEKE, MATTHEW (AG/1000)
<[EMAIL PROTECTED]> wrote:
> Is there any reason not to just use something as simple as drbd (which is baked into vanilla linux)? Unless you are already paying terracotta for their software there are plenty of HA setups that are supported through the use of drbd and a virtual ip.
>
> Matt
>
> -----Original Message-----
> From: Gokul Prasad [mailto:[EMAIL PROTECTED]]
> Sent: Friday, September 23, 2011 2:06 PM
> To: [EMAIL PROTECTED]
> Subject: Re: hadoop high availability using terracotta
>
> Cos,
>
> Am planning to implement this idea in bigger cluster but first i want
> to get confidence in small cluster. I do read about terracotta and
> inspired by this idea.
>
> -gokul
>
> On Fri, Sep 23, 2011 at 11:57 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
>> Gokul,
>>
>> Terracotta is a fascinating framework which might be a bit of an overshot for a
>> small setup as well as adding another level of complexity in the form
>> Terracotta framework.
>>
>> Quite interesting approach for NN's HA is Jini (aka ASF River) along with
>> JavaSpaces for the distributed storage of the meta-data. Using Jini also gives
>> you for free such benefits as auto-discovering making hot-swap of NN a non-issue.
>>
>> I've been musing over this idea for some time and am trying to find some time
>> to actually experiment with it.
>>
>> Cos
>>
>> On Fri, Sep 23, 2011 at 11:36PM, Gokul Prasad wrote:
>>>    Hi,
>>>    Am trying to bring high availability in namenode(in hadoop cluster) for
>>>    small setup.
>>>    So the idea is to keep one extra namenode and use terracotta to replicate
>>>    the log(in memory details-jvm) from primary datacode to standby namenode,
>>>    if primary goes down secondary can take the primary role.
>>>    Now am bit confused how to start second namenode while the primary name
>>>    node up and running(with datanode connected) with keep relying the info
>>>    from primary thro terracotta. Is there a way to achieve this without
>>>    restarting namenode or datanode, any patch?
>>>    Please help and suggest me.
>>>    Regards,
>>>    Gokul
>>
> This e-mail message may contain privileged and/or confidential information, and is intended to be received only by persons entitled
> to receive such information. If you have received this e-mail in error, please notify the sender immediately. Please delete it and
> all attachments from any servers, hard drives or any other media. Other use of this e-mail by you is strictly prohibited.
>
> All e-mails and attachments sent and received are subject to monitoring, reading and archival by Monsanto, including its
> subsidiaries. The recipient of this e-mail is solely responsible for checking for the presence of "Viruses" or other "Malware".
> Monsanto, along with its subsidiaries, accepts no liability for any damage caused by any such code transmitted by or accompanying
> this e-mail or any attachment.
>
>
> The information contained in this email may be subject to the export control laws and regulations of the United States, potentially
> including but not limited to the Export Administration Regulations (EAR) and sanctions regulations issued by the U.S. Department of
> Treasury, Office of Foreign Asset Controls (OFAC).  As a recipient of this information you are obligated to comply with all
> applicable U.S. export laws and regulations.
>
>

I don't have much terracotta experience but which solution of theirs are you referring to? After looking at their product page I am having a hard time figuring out what architecture you are planning to build that would have a faster turnaround on failure.

Matt

-----Original Message-----
From: Gokul Prasad [mailto:[EMAIL PROTECTED]]
Sent: Saturday, September 24, 2011 11:33 AM
To: [EMAIL PROTECTED]
Subject: Re: hadoop high availability using terracotta

afaik..drbd approach will take longer to bring standby namenode to
live..am trying to minimise the downtime..like standby namenode will
be relayed with the necessary info thro terracotta and if primary
namenode goes down standby will take its role.

-gokul

On Sat, Sep 24, 2011 at 1:00 AM, GOEKE, MATTHEW (AG/1000)
<[EMAIL PROTECTED]> wrote:
> Is there any reason not to just use something as simple as drbd (which is baked into vanilla linux)? Unless you are already paying terracotta for their software there are plenty of HA setups that are supported through the use of drbd and a virtual ip.
>
> Matt
>
> -----Original Message-----
> From: Gokul Prasad [mailto:[EMAIL PROTECTED]]
> Sent: Friday, September 23, 2011 2:06 PM
> To: [EMAIL PROTECTED]
> Subject: Re: hadoop high availability using terracotta
>
> Cos,
>
> Am planning to implement this idea in bigger cluster but first i want
> to get confidence in small cluster. I do read about terracotta and
> inspired by this idea.
>
> -gokul
>
> On Fri, Sep 23, 2011 at 11:57 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
>> Gokul,
>>
>> Terracotta is a fascinating framework which might be a bit of an overshot for a
>> small setup as well as adding another level of complexity in the form
>> Terracotta framework.
>>
>> Quite interesting approach for NN's HA is Jini (aka ASF River) along with
>> JavaSpaces for the distributed storage of the meta-data. Using Jini also gives
>> you for free such benefits as auto-discovering making hot-swap of NN a non-issue.
>>
>> I've been musing over this idea for some time and am trying to find some time
>> to actually experiment with it.
>>
>> Cos
>>
>> On Fri, Sep 23, 2011 at 11:36PM, Gokul Prasad wrote:
>>>    Hi,
>>>    Am trying to bring high availability in namenode(in hadoop cluster) for
>>>    small setup.
>>>    So the idea is to keep one extra namenode and use terracotta to replicate
>>>    the log(in memory details-jvm) from primary datacode to standby namenode,
>>>    if primary goes down secondary can take the primary role.
>>>    Now am bit confused how to start second namenode while the primary name
>>>    node up and running(with datanode connected) with keep relying the info
>>>    from primary thro terracotta. Is there a way to achieve this without
>>>    restarting namenode or datanode, any patch?
>>>    Please help and suggest me.
>>>    Regards,
>>>    Gokul
>>
> This e-mail message may contain privileged and/or confidential information, and is intended to be received only by persons entitled
> to receive such information. If you have received this e-mail in error, please notify the sender immediately. Please delete it and
> all attachments from any servers, hard drives or any other media. Other use of this e-mail by you is strictly prohibited.
>
> All e-mails and attachments sent and received are subject to monitoring, reading and archival by Monsanto, including its
> subsidiaries. The recipient of this e-mail is solely responsible for checking for the presence of "Viruses" or other "Malware".
> Monsanto, along with its subsidiaries, accepts no liability for any damage caused by any such code transmitted by or accompanying
> this e-mail or any attachment.
>
>
> The information contained in this email may be subject to the export control laws and regulations of the United States, potentially
> including but not limited to the Export Administration Regulations (EAR) and sanctions regulations issued by the U.S. Department of
> Treasury, Office of Foreign Asset Controls (OFAC).  As a recipient of this information you are obligated to comply with all