Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> Recovering fsImage from Namenode Logs


+
ishan chhabra 2012-12-20, 06:26
+
Harsh J 2012-12-20, 06:40
+
ishan chhabra 2012-12-20, 08:33
Copy link to this message
-
Re: Recovering fsImage from Namenode Logs
On Thu, Dec 20, 2012 at 12:33 AM, ishan chhabra <[EMAIL PROTECTED]> wrote:
> Unfortunately, the checkpoint image that I have has the deletes recorded. I
> cannot use it. I do have an image that is 15 days old, which I am currently
> running.
>
> I looked at the my logs and I have the filename, block allocated and
> generation stamp. Can you explain to me the importance of the generation
> stamp here? Since my hdfs cluster is operational with the old image and I
> am writing new data to it, the generation stamp must have been incremented
> beyond what it was 15 days ago. If I try to restore a block that we
> written, lets say, 13 days ago, there can be generation stamp collision.
> So, if I stop my cluster and make the new entries with generation stamp
> increments after what is currently in the namenode, will it be ok? Is the
> generation stamp stored somewhere in the datanode or the block stored in
> the datanode?

The generation stamp is stored by the datanode in the block directory,
as part of the .meta filename.

For example, if block -8546336708468389550 has genstamp 1002, you
would see something like this:

cmccabe@keter:/h> ls -l
/r/data1/current/BP-380817083-127.0.0.1-1356638793552/current/finalized/*8546336708468389550*
total 8
-rw-r--r-- 1 cmccabe users 2025 Dec 27 12:38 blk_-8546336708468389550
-rw-r--r-- 1 cmccabe users   23 Dec 27 12:38 blk_-8546336708468389550_1002.meta

cheers,
Colin
>
> Thanks for the clarifications.
>
> On Wed, Dec 19, 2012 at 10:40 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> proper INode entries of them to append/recreate your fsimage. They are
>
>
>
>
> --
> Thanks.
>
> Regards,
> Ishan Chhabra
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB