Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Recovering fsImage from Namenode Logs

Copy link to this message
Re: Recovering fsImage from Namenode Logs
On Thu, Dec 20, 2012 at 12:33 AM, ishan chhabra <[EMAIL PROTECTED]> wrote:
> Unfortunately, the checkpoint image that I have has the deletes recorded. I
> cannot use it. I do have an image that is 15 days old, which I am currently
> running.
> I looked at the my logs and I have the filename, block allocated and
> generation stamp. Can you explain to me the importance of the generation
> stamp here? Since my hdfs cluster is operational with the old image and I
> am writing new data to it, the generation stamp must have been incremented
> beyond what it was 15 days ago. If I try to restore a block that we
> written, lets say, 13 days ago, there can be generation stamp collision.
> So, if I stop my cluster and make the new entries with generation stamp
> increments after what is currently in the namenode, will it be ok? Is the
> generation stamp stored somewhere in the datanode or the block stored in
> the datanode?

The generation stamp is stored by the datanode in the block directory,
as part of the .meta filename.

For example, if block -8546336708468389550 has genstamp 1002, you
would see something like this:

cmccabe@keter:/h> ls -l
total 8
-rw-r--r-- 1 cmccabe users 2025 Dec 27 12:38 blk_-8546336708468389550
-rw-r--r-- 1 cmccabe users   23 Dec 27 12:38 blk_-8546336708468389550_1002.meta

> Thanks for the clarifications.
> On Wed, Dec 19, 2012 at 10:40 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>> proper INode entries of them to append/recreate your fsimage. They are
> --
> Thanks.
> Regards,
> Ishan Chhabra