Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> HDFS-903 - Backupnode always downloading image from Namenode after this change


+
Sreehari G 2011-05-25, 08:08
+
Todd Lipcon 2011-05-25, 17:45
Copy link to this message
-
Re: HDFS-903 - Backupnode always downloading image from Namenode after this change
Hi Sreehari,

I actually also see this problem. It would be good to fix it.
You can simply create a jira describing the issue and we can discuss
there how to fix it. If you have a patch - even better.
Once fixed in trunk it can be ported to other branches,
including 0.22 or HDFS-1073-branch if desired.

Thanks,
--Konstantin

On Wed, May 25, 2011 at 10:45 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> Hi Sreehari,
>
> The BackupNode isn't really production-ized. I don't think anyone uses it,
> so it's not surprising that there are a lot of broken parts.
>
> If you're interested in contributing in this area, I would encourage you to
> look at the HDFS-1073 branch. The BN and CheckpointNode will be undergoing
> various surgery during the course of this project, so it's a good
> opportunity to fix it up, add test coverage, etc.
>
> -Todd
>
> On Wed, May 25, 2011 at 1:08 AM, Sreehari G <[EMAIL PROTECTED]> wrote:
>
> >  Hi all ,
> >
> > In HDFS-903 - ( md5 verification of fsimage ) , with this change ,
> > Backupnode is downloading the image & edit files from namenode everytime
> > since the difference in checkpoint time is always maintined b/w Namenode
> and
> > Backupnode . This happens since Namenode is resetting its checkpoint time
> > everytime since we are ignoring renewCheckpointTime and passing true
> > explicitly to rollFsimage during endcheckpoint ..
> >
> > Also , though a proposal for using md5 to decide whether to download
> image
> > is also mentioned , it doesnt seeem to be implemented ..
> >
> > Isn't this downloading of image everytime a problem or am I missing
> > something ?
> >
> > HUAWEI TECHNOLOGIES CO.,LTD. [image: huawei_logo]
> >
> >
> > Solitaire
> > Domlur
> > Bangalore
> > www.huawei.com
> >
> >
> -------------------------------------------------------------------------------------------------------------------------------------
> > This e-mail and its attachments contain confidential information from
> > HUAWEI, which
> > is intended only for the person or entity whose address is listed above.
> > Any use of the
> > information contained herein in any way (including, but not limited to,
> > total or partial
> > disclosure, reproduction, or dissemination) by persons other than the
> > intended
> > recipient(s) is prohibited. If you receive this e-mail in error, please
> > notify the sender by
> > phone or email immediately and delete it!
> >
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
+
Sreehari G 2011-05-27, 10:35
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB