Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> corrupted edits log after power failure

Gabi Kazav 2011-09-22, 08:48
Copy link to this message
Re: corrupted edits log after power failure
Does the backup process include syncing? On-drive write cache can also trick you.
For absolutely critical data, it is a good idea to use a controller with battery-backed write cache or a service/product that guarantees durability.


On 9/22/11 3:48 AM, "Gabi Kazav" <[EMAIL PROTECTED]> wrote:


I had Power Failure.
I have backup of files: edits, fsimage.

I am backing it up with:

curl -s http://nameNode:50070/getimage?getimage=1 > fsimage
curl -s http://nameNode:50070/getimage?getedits=1 > edits

When I am trying to start the HDFS with the recovered files, I got error about the edits file : "Error replaying edit log at offset 1921"

Also, I have edits.new file, when I rename it to edits I got: "ERROR org.apache.hadoop.hdfs.server.common.Storage: Error replaying edit log at offset 2494103"

What is the problem?!
And from now on, how can I do a backup that works?! :)

Gabi Kazav
IT Manager And Infrastructure Engineer
[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> | www.pursway.com<http://www.pursway.com/>
Mailing address PO Box 4125, Herzliya 46140
Address 8 Hamada St., Herzliya, IL | Tel +972 527 772457| Fax + 972 9 958 4736
Brian Bockelman 2011-09-22, 19:15
Steve Loughran 2011-09-26, 14:34