Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> snapshotexport to outside of cluster


Copy link to this message
-
Re: snapshotexport to outside of cluster
We want to use that exported data in case of any disaster happens.

Thanks
-OC
On Sun, Dec 1, 2013 at 2:25 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Since the remote server doesn't have hdfs, what would the backup be used
> for ?
>
> Cheers
>
> On Nov 30, 2013, at 10:34 PM, oc tsdb <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> >>> Can you clarify what 'without copying to local file system' means ?
> >
> > We want to take HBase tables backup.So we periodically take snapshots and
> > export data to backup server which is not in cluster.
> >
> > Currently we are achieving this as mentioned below.
> >
> > 1.Create snapshots from HBase command line
> >   lets say we have created snapshot for a table called 'hbase_tabel' and
> > its snapshot name is 'hbase_tabel_snapshot'
> > 2. Then export snapshot data to remote backup server which is not in
> > cluster.We are achieving this in two steps.
> >   a. Export snapshot to local file system(sample path /tmp/hbase_backup)
> > from hdfs file system as mentioned below
> >       hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> > hbase_tabel_snapshot -copy-to file:///tmp/hbase_backup -
> >       mappers      16
> >      Here we should have enough disc space in local file system to export
> > snapshot from hdfs.
> >    b.Transfer(scp) exported snapshot from local file system
> > (/tmp/hbase_backup)
> > to backup server which is not hadoop/HBase running
> >       server.
> >
> >  That means first we have to export snapshot to local system from hdfs
> and
> > then transfer ( scp ) to remote server.Instead of doing in two steps can
> we
> > directly export snapshot to remote server(which is not in cluster) like
> how
> > it is exporting to another cluster.
> > .
> > Thanks
> > OC
> >
> >
> >
> >
> >
> > On Sat, Nov 30, 2013 at 11:11 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> >> HBASE-4618 is not related to snapshots.
> >> Lars did some work in HBASE-5509, a sub-task of HBASE-4618.
> >>
> >> FYI
> >>
> >>
> >> On Fri, Nov 29, 2013 at 9:31 PM, Yu Li <[EMAIL PROTECTED]> wrote:
> >>
> >>> I guess this might be a use case of
> >>> HBASE-7912<https://issues.apache.org/jira/browse/HBASE-7912>or
> >>> HBASE-4618 <https://issues.apache.org/jira/browse/HBASE-4618>? Maybe
> >> some
> >>> backup/restore tools taking use of snapshot would be nice to have?
> >>>
> >>>
> >>> On 30 November 2013 12:58, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> The remote server is in different cluster, right ?
> >>>>
> >>>> Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies
> >>> data
> >>>> to outputPath which is in the other cluster.
> >>>>
> >>>> Can you clarify what 'without copying to local file system' means ?
> >>>>
> >>>> Cheers
> >>>>
> >>>>
> >>>> On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> Hi,
> >>>>> thanks for your replay.
> >>>>>
> >>>>> We are able to export snapshot to another cluster But this is not
> >> what
> >>> we
> >>>>> are looking for.
> >>>>> We want to export snapshot to remote server without copying to local
> >>> file
> >>>>> system.
> >>>>>
> >>>>> Thanks
> >>>>> OC
> >>>>>
> >>>>> On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>>>>
> >>>>>> Here is a sample command exporting to another cluster:
> >>>>>>
> >>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> >>>>>> gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb <[EMAIL PROTECTED]>
> >> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> can someone please let us know if we can export snapshots data
> >>>> directly
> >>>>>> to
> >>>>>>> outside of cluster without copying to local.
> >>>>>>>
> >>>>>>> Please refer us if there are any blog with example.
> >>>>>>>
> >>>>>>> Below command is copying data to local file system under /tmp.
> >>>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> >>>> mytable
> >>>>>>> -copy-to file:///tmp/hbase_backup -mappers 16