It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in 1.0.1?

Thanks
-jeremy

BackupNode is major functionality with change in required in RPC protocols,
configuration etc. Hence it will not be available in bug fix release 1.0.1.

It is also unlikely to be not available on minor releases in 1.x release
streams.

Regards,
Suresh

On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:

>
> It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in
> 1.0.1?
>
> Thanks
> -jeremy

Any possibility of getting spec files to create packages for 0.22?

Thanks
-jeremy

On Feb 22, 2012, at 11:50 AM, Suresh Srinivas wrote:

> BackupNode is major functionality with change in required in RPC protocols,
> configuration etc. Hence it will not be available in bug fix release 1.0.1.
>
> It is also unlikely to be not available on minor releases in 1.x release
> streams.
>
> Regards,
> Suresh
>
> On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>
>>
>> It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in
>> 1.0.1?
>>
>> Thanks
>> -jeremy

Check out the Apache Bigtop project. I believe they have 0.22 RPMs.

Out of curiosity, why are you interested in BackupNode?

-Joey

Sent from my iPhone

On Feb 22, 2012, at 14:56, Jeremy Hansen <[EMAIL PROTECTED]> wrote:

> Any possibility of getting spec files to create packages for 0.22?
>
> Thanks
> -jeremy
>
> On Feb 22, 2012, at 11:50 AM, Suresh Srinivas wrote:
>
>> BackupNode is major functionality with change in required in RPC protocols,
>> configuration etc. Hence it will not be available in bug fix release 1.0.1.
>>
>> It is also unlikely to be not available on minor releases in 1.x release
>> streams.
>>
>> Regards,
>> Suresh
>>
>> On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>
>>>
>>> It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in
>>> 1.0.1?
>>>
>>> Thanks
>>> -jeremy
>

I guess I thought that backupnode would provide some level of namenode redundancy.  Perhaps I don't fully understand.

I'll check out Bigtop.  I looked at it a while ago and forgot about it.

Thanks
-jeremy

On Feb 22, 2012, at 2:43 PM, Joey Echeverria wrote:

> Check out the Apache Bigtop project. I believe they have 0.22 RPMs.
>
> Out of curiosity, why are you interested in BackupNode?
>
> -Joey
>
> Sent from my iPhone
>
> On Feb 22, 2012, at 14:56, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>
>> Any possibility of getting spec files to create packages for 0.22?
>>
>> Thanks
>> -jeremy
>>
>> On Feb 22, 2012, at 11:50 AM, Suresh Srinivas wrote:
>>
>>> BackupNode is major functionality with change in required in RPC protocols,
>>> configuration etc. Hence it will not be available in bug fix release 1.0.1.
>>>
>>> It is also unlikely to be not available on minor releases in 1.x release
>>> streams.
>>>
>>> Regards,
>>> Suresh
>>>
>>> On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>>
>>>>
>>>> It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in
>>>> 1.0.1?
>>>>
>>>> Thanks
>>>> -jeremy
>>

By the way, I don't see anything 0.22 based in the bigtop repos.

Thanks
-jeremy

On Feb 22, 2012, at 3:58 PM, Jeremy Hansen wrote:

> I guess I thought that backupnode would provide some level of namenode redundancy.  Perhaps I don't fully understand.
>
> I'll check out Bigtop.  I looked at it a while ago and forgot about it.
>
> Thanks
> -jeremy
>
> On Feb 22, 2012, at 2:43 PM, Joey Echeverria wrote:
>
>> Check out the Apache Bigtop project. I believe they have 0.22 RPMs.
>>
>> Out of curiosity, why are you interested in BackupNode?
>>
>> -Joey
>>
>> Sent from my iPhone
>>
>> On Feb 22, 2012, at 14:56, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>
>>> Any possibility of getting spec files to create packages for 0.22?
>>>
>>> Thanks
>>> -jeremy
>>>
>>> On Feb 22, 2012, at 11:50 AM, Suresh Srinivas wrote:
>>>
>>>> BackupNode is major functionality with change in required in RPC protocols,
>>>> configuration etc. Hence it will not be available in bug fix release 1.0.1.
>>>>
>>>> It is also unlikely to be not available on minor releases in 1.x release
>>>> streams.
>>>>
>>>> Regards,
>>>> Suresh
>>>>
>>>> On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>>>
>>>>>
>>>>> It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in
>>>>> 1.0.1?
>>>>>
>>>>> Thanks
>>>>> -jeremy
>>>
>

Check out this branch for the 0.22 version of Bigtop:

https://svn.apache.org/repos/asf/incubator/bigtop/branches/hadoop-0.22/

However, I don't think BackupNode is what you want. It sounds like you
want HA which is coming in (hopefully) 0.23.2 and is also available
today in CDH4b1.

-Joey

On Wed, Feb 22, 2012 at 7:09 PM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
> By the way, I don't see anything 0.22 based in the bigtop repos.
>
> Thanks
> -jeremy
>
> On Feb 22, 2012, at 3:58 PM, Jeremy Hansen wrote:
>
>> I guess I thought that backupnode would provide some level of namenode redundancy.  Perhaps I don't fully understand.
>>
>> I'll check out Bigtop.  I looked at it a while ago and forgot about it.
>>
>> Thanks
>> -jeremy
>>
>> On Feb 22, 2012, at 2:43 PM, Joey Echeverria wrote:
>>
>>> Check out the Apache Bigtop project. I believe they have 0.22 RPMs.
>>>
>>> Out of curiosity, why are you interested in BackupNode?
>>>
>>> -Joey
>>>
>>> Sent from my iPhone
>>>
>>> On Feb 22, 2012, at 14:56, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>>
>>>> Any possibility of getting spec files to create packages for 0.22?
>>>>
>>>> Thanks
>>>> -jeremy
>>>>
>>>> On Feb 22, 2012, at 11:50 AM, Suresh Srinivas wrote:
>>>>
>>>>> BackupNode is major functionality with change in required in RPC protocols,
>>>>> configuration etc. Hence it will not be available in bug fix release 1.0.1.
>>>>>
>>>>> It is also unlikely to be not available on minor releases in 1.x release
>>>>> streams.
>>>>>
>>>>> Regards,
>>>>> Suresh
>>>>>
>>>>> On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>>
>>>>>> It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in
>>>>>> 1.0.1?
>>>>>>
>>>>>> Thanks
>>>>>> -jeremy
>>>>
>>
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434

Joey,

Can you please answer the question from in the context of Apache releases.
Not sure if CDH4b1 needs to be mentioned in the context of this mailing
list.

Regards,
Suresh

On Wed, Feb 22, 2012 at 5:24 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:

> Check out this branch for the 0.22 version of Bigtop:
>
> https://svn.apache.org/repos/asf/incubator/bigtop/branches/hadoop-0.22/
>
> However, I don't think BackupNode is what you want. It sounds like you
> want HA which is coming in (hopefully) 0.23.2 and is also available
> today in CDH4b1.
>
> -Joey
>
> On Wed, Feb 22, 2012 at 7:09 PM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
> > By the way, I don't see anything 0.22 based in the bigtop repos.
> >
> > Thanks
> > -jeremy
> >
> > On Feb 22, 2012, at 3:58 PM, Jeremy Hansen wrote:
> >
> >> I guess I thought that backupnode would provide some level of namenode
> redundancy.  Perhaps I don't fully understand.
> >>
> >> I'll check out Bigtop.  I looked at it a while ago and forgot about it.
> >>
> >> Thanks
> >> -jeremy
> >>
> >> On Feb 22, 2012, at 2:43 PM, Joey Echeverria wrote:
> >>
> >>> Check out the Apache Bigtop project. I believe they have 0.22 RPMs.
> >>>
> >>> Out of curiosity, why are you interested in BackupNode?
> >>>
> >>> -Joey
> >>>
> >>> Sent from my iPhone
> >>>
> >>> On Feb 22, 2012, at 14:56, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> Any possibility of getting spec files to create packages for 0.22?
> >>>>
> >>>> Thanks
> >>>> -jeremy
> >>>>
> >>>> On Feb 22, 2012, at 11:50 AM, Suresh Srinivas wrote:
> >>>>
> >>>>> BackupNode is major functionality with change in required in RPC
> protocols,
> >>>>> configuration etc. Hence it will not be available in bug fix release
> 1.0.1.
> >>>>>
> >>>>> It is also unlikely to be not available on minor releases in 1.x
> release
> >>>>> streams.
> >>>>>
> >>>>> Regards,
> >>>>> Suresh
> >>>>>
> >>>>> On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]>
> wrote:
> >>>>>
> >>>>>>
> >>>>>> It looks as if backupnode isn't supported in 1.0.0?  Any chances
> it's in
> >>>>>> 1.0.1?
> >>>>>>
> >>>>>> Thanks
> >>>>>> -jeremy
> >>>>
> >>
> >
>
>
>
> --
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434
>

Thanks.  Could you clarify what BackupNode does?

-jeremy

On Feb 22, 2012, at 5:24 PM, Joey Echeverria wrote:

> Check out this branch for the 0.22 version of Bigtop:
>
> https://svn.apache.org/repos/asf/incubator/bigtop/branches/hadoop-0.22/
>
> However, I don't think BackupNode is what you want. It sounds like you
> want HA which is coming in (hopefully) 0.23.2 and is also available
> today in CDH4b1.
>
> -Joey
>
> On Wed, Feb 22, 2012 at 7:09 PM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>> By the way, I don't see anything 0.22 based in the bigtop repos.
>>
>> Thanks
>> -jeremy
>>
>> On Feb 22, 2012, at 3:58 PM, Jeremy Hansen wrote:
>>
>>> I guess I thought that backupnode would provide some level of namenode redundancy.  Perhaps I don't fully understand.
>>>
>>> I'll check out Bigtop.  I looked at it a while ago and forgot about it.
>>>
>>> Thanks
>>> -jeremy
>>>
>>> On Feb 22, 2012, at 2:43 PM, Joey Echeverria wrote:
>>>
>>>> Check out the Apache Bigtop project. I believe they have 0.22 RPMs.
>>>>
>>>> Out of curiosity, why are you interested in BackupNode?
>>>>
>>>> -Joey
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Feb 22, 2012, at 14:56, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> Any possibility of getting spec files to create packages for 0.22?
>>>>>
>>>>> Thanks
>>>>> -jeremy
>>>>>
>>>>> On Feb 22, 2012, at 11:50 AM, Suresh Srinivas wrote:
>>>>>
>>>>>> BackupNode is major functionality with change in required in RPC protocols,
>>>>>> configuration etc. Hence it will not be available in bug fix release 1.0.1.
>>>>>>
>>>>>> It is also unlikely to be not available on minor releases in 1.x release
>>>>>> streams.
>>>>>>
>>>>>> Regards,
>>>>>> Suresh
>>>>>>
>>>>>> On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>>
>>>>>>> It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in
>>>>>>> 1.0.1?
>>>>>>>
>>>>>>> Thanks
>>>>>>> -jeremy
>>>>>
>>>
>>
>
>
>
> --
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434

Hi Jeremy,

You can find the design doc for the BackupNode, via HADOOP-4539, at
https://issues.apache.org/jira/secure/attachment/12400631/StreamEditsToBN.pdf

On Thu, Feb 23, 2012 at 2:11 PM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
> Thanks.  Could you clarify what BackupNode does?
>
> -jeremy
>
> On Feb 22, 2012, at 5:24 PM, Joey Echeverria wrote:
>
>> Check out this branch for the 0.22 version of Bigtop:
>>
>> https://svn.apache.org/repos/asf/incubator/bigtop/branches/hadoop-0.22/
>>
>> However, I don't think BackupNode is what you want. It sounds like you
>> want HA which is coming in (hopefully) 0.23.2 and is also available
>> today in CDH4b1.
>>
>> -Joey
>>
>> On Wed, Feb 22, 2012 at 7:09 PM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>> By the way, I don't see anything 0.22 based in the bigtop repos.
>>>
>>> Thanks
>>> -jeremy
>>>
>>> On Feb 22, 2012, at 3:58 PM, Jeremy Hansen wrote:
>>>
>>>> I guess I thought that backupnode would provide some level of namenode redundancy.  Perhaps I don't fully understand.
>>>>
>>>> I'll check out Bigtop.  I looked at it a while ago and forgot about it.
>>>>
>>>> Thanks
>>>> -jeremy
>>>>
>>>> On Feb 22, 2012, at 2:43 PM, Joey Echeverria wrote:
>>>>
>>>>> Check out the Apache Bigtop project. I believe they have 0.22 RPMs.
>>>>>
>>>>> Out of curiosity, why are you interested in BackupNode?
>>>>>
>>>>> -Joey
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Feb 22, 2012, at 14:56, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> Any possibility of getting spec files to create packages for 0.22?
>>>>>>
>>>>>> Thanks
>>>>>> -jeremy
>>>>>>
>>>>>> On Feb 22, 2012, at 11:50 AM, Suresh Srinivas wrote:
>>>>>>
>>>>>>> BackupNode is major functionality with change in required in RPC protocols,
>>>>>>> configuration etc. Hence it will not be available in bug fix release 1.0.1.
>>>>>>>
>>>>>>> It is also unlikely to be not available on minor releases in 1.x release
>>>>>>> streams.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Suresh
>>>>>>>
>>>>>>> On Wed, Feb 22, 2012 at 11:40 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> It looks as if backupnode isn't supported in 1.0.0?  Any chances it's in
>>>>>>>> 1.0.1?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -jeremy
>>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Joseph Echeverria
>> Cloudera, Inc.
>> 443.305.9434
>

--
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about

On Thu, Feb 23, 2012 at 12:41 AM, Jeremy Hansen <[EMAIL PROTECTED]> wrote:

> Thanks.  Could you clarify what BackupNode does?
>
> -jeremy
>
>
Namenode currently keeps the entire file system namespace in memory. It
logs the write operations (create, delete file etc.) into a journal file
called editlog. This journal needs to be merged with the file system image
periodically to avoid journal file growing to a large size. This is called
checkpointing. Checkpoint also reduces the startup time, since the namenode
need not load large editlog file.

Prior to release 0.21, another node called SecondaryNamenode was used  for
checkpointing. It periodically gets the file system image and edit, load it
into memory and write checkpoint image. This image is then then shipped to
the Namenode.

In 0.21, BackupNode was introduced. Unlike SecondaryNamenode, it gets edits
streamed from the Namenode. It periodically writes the checkpoint image and
ships it back to Namenode. The goal was for this to become Standby node,
towards Namenode HA. Konstantin and few others are pursuing this.

I have not seen any deployments of BackupNode in production. I would love
to hear if any one has deployed it in production and how stable it is.

Regards,
Suresh