Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Snapshot clone error


Copy link to this message
-
Re: Snapshot clone error
It ended up working. Thank you for your help.

/David
On Fri, Jan 31, 2014 at 4:46 PM, Matteo Bertozzi <[EMAIL PROTECTED]>wrote:

> thanks for the confirmation.
>
> can you try to export the snapshot again and keep the log file if the
> result of the export will be broken again?
> Thanks!
>
> Matteo
>
>
>
> On Fri, Jan 31, 2014 at 3:43 PM, David Koch <[EMAIL PROTECTED]> wrote:
>
> > Actually, I just noticed - the snapshot on the source cluster ok, it's
> the
> > exported snapshot on the destination cluster that's corrupted.
> >
> >
> > On Fri, Jan 31, 2014 at 4:40 PM, David Koch <[EMAIL PROTECTED]>
> wrote:
> >
> > > Thanks for your reply,
> > >
> > > As a matter of fact when running with the "-files" option it turns out
> a
> > > lot of files are missing from the snapshot which I did not manage to
> > > restore. It's possible that hbck was run during snapshotting.
> > >
> > > **************************************************************
> > > BAD SNAPSHOT: 6659 hfile(s) and 0 log(s) missing.
> > > **************************************************************
> > > 78 HFiles (78 in archive), total size 14.3 G (0.00% 0 shared with the
> > > source table)
> > > 0 Logs, total size 0
> > >
> > > 78 files is exactly the number of regions that I found after attempting
> > > restoration.
> > >
> > > We followed standard procedure as described in the manual:
> > > http://hbase.apache.org/book/ops.snapshots.html
> > >
> > > I will try again and make sure no hbchk is intervening.
> > >
> > > /David
> > >
> > >
> > > On Fri, Jan 31, 2014 at 4:20 PM, Matteo Bertozzi <
> > [EMAIL PROTECTED]>wrote:
> > >
> > >> you should use SnapshotInfo with the "-files" options and you'll
> > probably
> > >> see that one snapshot is corrupted.
> > >> in HBase 0.94.15/CDH 4.6 there will be a fix (HBASE-10111) that will
> > >> prevent to restore/clone a corrupted snapshot.
> > >>
> > >> a corrupted snapshot means that some file contained in the snapshot is
> > >> missing from the .archive
> > >> that situation may happen if you have removed files by hand, or you
> run
> > >> hbck that sidelined the files or similar
> > >> (unless there is a bug somewhere)
> > >> do you remember the steps that you followed? did you use
> ExportSnapshot?
> > >> did you moved the files by hand to another cluster or similar?
> > >>
> > >> Offline or Online snapshot shouldn't make difference, the corruption
> is
> > >> probably happened after taking the snapshot.
> > >> You can retry taking the snapshot, and periodically run SnapshotInfo
> > with
> > >> the -files options to verify the state and post the logs in case you
> > get a
> > >> corruption again.
> > >>
> > >> Matteo
> > >>
> > >>
> > >>
> > >> On Fri, Jan 31, 2014 at 3:10 PM, David Koch <[EMAIL PROTECTED]>
> > >> wrote:
> > >>
> > >> > Matteo,
> > >> >
> > >> > Thank you for your reply. All clients, servers are using the same
> > >> version:
> > >> >
> > >> > 14/01/31 16:06:20 INFO util.VersionInfo: HBase 0.94.6-cdh4.5.0
> > >> >
> > >> > Also, the information generated by:
> > >> >
> > >> > hbase org.apache.hadoop.hbase.snapshot.SnapshotInfo -snapshot
> > >> >
> > >> > is identical for snapshots which I managed to clone and those for
> > which
> > >> the
> > >> > cloning/restoration failed. Would you advise re-trying snaphotting
> the
> > >> > table while it is disabled? Otherwise I'll go with old-fashioned
> > >> CopyTable
> > >> > or re-import into HBase from HDFS files.
> > >> >
> > >> > Thank you,
> > >> >
> > >> > /David
> > >> >
> > >> >
> > >> > On Fri, Jan 31, 2014 at 2:42 PM, Matteo Bertozzi <
> > >> [EMAIL PROTECTED]
> > >> > >wrote:
> > >> >
> > >> > > the snapshot seems to be corrupted, which version are you running?
> > >> > >
> > >> > > Matteo
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Fri, Jan 31, 2014 at 1:06 PM, David Koch <
> [EMAIL PROTECTED]>
> > >> > wrote:
> > >> > >
> > >> > > > Hello,
> > >> > > >
> > >> > > > We export an online snapshot of a table to a different cluster,

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB