Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> corrupted edits log after power failure


+
Gabi Kazav 2011-09-22, 08:48
Copy link to this message
-
Re: corrupted edits log after power failure
Does the backup process include syncing? On-drive write cache can also trick you.
For absolutely critical data, it is a good idea to use a controller with battery-backed write cache or a service/product that guarantees durability.

Kihwal

On 9/22/11 3:48 AM, "Gabi Kazav" <[EMAIL PROTECTED]> wrote:

Hi,

I had Power Failure.
I have backup of files: edits, fsimage.

I am backing it up with:

curl -s http://nameNode:50070/getimage?getimage=1 > fsimage
curl -s http://nameNode:50070/getimage?getedits=1 > edits

When I am trying to start the HDFS with the recovered files, I got error about the edits file : "Error replaying edit log at offset 1921"

Also, I have edits.new file, when I rename it to edits I got: "ERROR org.apache.hadoop.hdfs.server.common.Storage: Error replaying edit log at offset 2494103"

What is the problem?!
And from now on, how can I do a backup that works?! :)

Thanks,
Gabi.
Gabi Kazav
IT Manager And Infrastructure Engineer
[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> | www.pursway.com<http://www.pursway.com/>
Mailing address PO Box 4125, Herzliya 46140
Address 8 Hamada St., Herzliya, IL | Tel +972 527 772457| Fax + 972 9 958 4736
+
Brian Bockelman 2011-09-22, 19:15
+
Steve Loughran 2011-09-26, 14:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB