Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # dev - corrupted edits log after power failure


+
Gabi Kazav 2011-09-22, 08:48
Copy link to this message
-
Re: corrupted edits log after power failure
Kihwal Lee 2011-09-22, 17:37
Does the backup process include syncing? On-drive write cache can also trick you.
For absolutely critical data, it is a good idea to use a controller with battery-backed write cache or a service/product that guarantees durability.

Kihwal

On 9/22/11 3:48 AM, "Gabi Kazav" <[EMAIL PROTECTED]> wrote:

Hi,

I had Power Failure.
I have backup of files: edits, fsimage.

I am backing it up with:

curl -s http://nameNode:50070/getimage?getimage=1 > fsimage
curl -s http://nameNode:50070/getimage?getedits=1 > edits

When I am trying to start the HDFS with the recovered files, I got error about the edits file : "Error replaying edit log at offset 1921"

Also, I have edits.new file, when I rename it to edits I got: "ERROR org.apache.hadoop.hdfs.server.common.Storage: Error replaying edit log at offset 2494103"

What is the problem?!
And from now on, how can I do a backup that works?! :)

Thanks,
Gabi.
Gabi Kazav
IT Manager And Infrastructure Engineer
[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> | www.pursway.com<http://www.pursway.com/>
Mailing address PO Box 4125, Herzliya 46140
Address 8 Hamada St., Herzliya, IL | Tel +972 527 772457| Fax + 972 9 958 4736
+
Brian Bockelman 2011-09-22, 19:15
+
Steve Loughran 2011-09-26, 14:34