Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Snapshot clone error


Copy link to this message
-
Re: Snapshot clone error
It ended up working. Thank you for your help.

/David
On Fri, Jan 31, 2014 at 4:46 PM, Matteo Bertozzi <[EMAIL PROTECTED]>wrote:

> thanks for the confirmation.
>
> can you try to export the snapshot again and keep the log file if the
> result of the export will be broken again?
> Thanks!
>
> Matteo
>
>
>
> On Fri, Jan 31, 2014 at 3:43 PM, David Koch <[EMAIL PROTECTED]> wrote:
>
> > Actually, I just noticed - the snapshot on the source cluster ok, it's
> the
> > exported snapshot on the destination cluster that's corrupted.
> >
> >
> > On Fri, Jan 31, 2014 at 4:40 PM, David Koch <[EMAIL PROTECTED]>
> wrote:
> >
> > > Thanks for your reply,
> > >
> > > As a matter of fact when running with the "-files" option it turns out
> a
> > > lot of files are missing from the snapshot which I did not manage to
> > > restore. It's possible that hbck was run during snapshotting.
> > >
> > > **************************************************************
> > > BAD SNAPSHOT: 6659 hfile(s) and 0 log(s) missing.
> > > **************************************************************
> > > 78 HFiles (78 in archive), total size 14.3 G (0.00% 0 shared with the
> > > source table)
> > > 0 Logs, total size 0
> > >
> > > 78 files is exactly the number of regions that I found after attempting
> > > restoration.
> > >
> > > We followed standard procedure as described in the manual:
> > > http://hbase.apache.org/book/ops.snapshots.html
> > >
> > > I will try again and make sure no hbchk is intervening.
> > >
> > > /David
> > >
> > >
> > > On Fri, Jan 31, 2014 at 4:20 PM, Matteo Bertozzi <
> > [EMAIL PROTECTED]>wrote:
> > >
> > >> you should use SnapshotInfo with the "-files" options and you'll
> > probably
> > >> see that one snapshot is corrupted.
> > >> in HBase 0.94.15/CDH 4.6 there will be a fix (HBASE-10111) that will
> > >> prevent to restore/clone a corrupted snapshot.
> > >>
> > >> a corrupted snapshot means that some file contained in the snapshot is
> > >> missing from the .archive
> > >> that situation may happen if you have removed files by hand, or you
> run
> > >> hbck that sidelined the files or similar
> > >> (unless there is a bug somewhere)
> > >> do you remember the steps that you followed? did you use
> ExportSnapshot?
> > >> did you moved the files by hand to another cluster or similar?
> > >>
> > >> Offline or Online snapshot shouldn't make difference, the corruption
> is
> > >> probably happened after taking the snapshot.
> > >> You can retry taking the snapshot, and periodically run SnapshotInfo
> > with
> > >> the -files options to verify the state and post the logs in case you
> > get a
> > >> corruption again.
> > >>
> > >> Matteo
> > >>
> > >>
> > >>
> > >> On Fri, Jan 31, 2014 at 3:10 PM, David Koch <[EMAIL PROTECTED]>
> > >> wrote:
> > >>
> > >> > Matteo,
> > >> >
> > >> > Thank you for your reply. All clients, servers are using the same
> > >> version:
> > >> >
> > >> > 14/01/31 16:06:20 INFO util.VersionInfo: HBase 0.94.6-cdh4.5.0
> > >> >
> > >> > Also, the information generated by:
> > >> >
> > >> > hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
> > >> >
> > >> > is identical for snapshots which I managed to clone and those for
> > which
> > >> the
> > >> > cloning/restoration failed. Would you advise re-trying snaphotting
> the
> > >> > table while it is disabled? Otherwise I'll go with old-fashioned
> > >> CopyTable
> > >> > or re-import into HBase from HDFS files.
> > >> >
> > >> > Thank you,
> > >> >
> > >> > /David
> > >> >
> > >> >
> > >> > On Fri, Jan 31, 2014 at 2:42 PM, Matteo Bertozzi <
> > >> [EMAIL PROTECTED]
> > >> > >wrote:
> > >> >
> > >> > > the snapshot seems to be corrupted, which version are you running?
> > >> > >
> > >> > > Matteo
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Fri, Jan 31, 2014 at 1:06 PM, David Koch <
> [EMAIL PROTECTED]>
> > >> > wrote:
> > >> > >
> > >> > > > Hello,
> > >> > > >
> > >> > > > We export an online snapshot of a table to a different cluster,