Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> snapshotexport to outside of cluster


Copy link to this message
-
Re: snapshotexport to outside of cluster
We want to use that exported data in case of any disaster happens.

Thanks
-OC
On Sun, Dec 1, 2013 at 2:25 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Since the remote server doesn't have hdfs, what would the backup be used
> for ?
>
> Cheers
>
> On Nov 30, 2013, at 10:34 PM, oc tsdb <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> >>> Can you clarify what 'without copying to local file system' means ?
> >
> > We want to take HBase tables backup.So we periodically take snapshots and
> > export data to backup server which is not in cluster.
> >
> > Currently we are achieving this as mentioned below.
> >
> > 1.Create snapshots from HBase command line
> >   lets say we have created snapshot for a table called 'hbase_tabel' and
> > its snapshot name is 'hbase_tabel_snapshot'
> > 2. Then export snapshot data to remote backup server which is not in
> > cluster.We are achieving this in two steps.
> >   a. Export snapshot to local file system(sample path /tmp/hbase_backup)
> > from hdfs file system as mentioned below
> >       hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> > hbase_tabel_snapshot -copy-to file:///tmp/hbase_backup -
> >       mappers      16
> >      Here we should have enough disc space in local file system to export
> > snapshot from hdfs.
> >    b.Transfer(scp) exported snapshot from local file system
> > (/tmp/hbase_backup)
> > to backup server which is not hadoop/HBase running
> >       server.
> >
> >  That means first we have to export snapshot to local system from hdfs
> and
> > then transfer ( scp ) to remote server.Instead of doing in two steps can
> we
> > directly export snapshot to remote server(which is not in cluster) like
> how
> > it is exporting to another cluster.
> > .
> > Thanks
> > OC
> >
> >
> >
> >
> >
> > On Sat, Nov 30, 2013 at 11:11 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> >> HBASE-4618 is not related to snapshots.
> >> Lars did some work in HBASE-5509, a sub-task of HBASE-4618.
> >>
> >> FYI
> >>
> >>
> >> On Fri, Nov 29, 2013 at 9:31 PM, Yu Li <[EMAIL PROTECTED]> wrote:
> >>
> >>> I guess this might be a use case of
> >>> HBASE-7912<https://issues.apache.org/jira/browse/HBASE-7912>or
> >>> HBASE-4618 <https://issues.apache.org/jira/browse/HBASE-4618>? Maybe
> >> some
> >>> backup/restore tools taking use of snapshot would be nice to have?
> >>>
> >>>
> >>> On 30 November 2013 12:58, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> The remote server is in different cluster, right ?
> >>>>
> >>>> Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies
> >>> data
> >>>> to outputPath which is in the other cluster.
> >>>>
> >>>> Can you clarify what 'without copying to local file system' means ?
> >>>>
> >>>> Cheers
> >>>>
> >>>>
> >>>> On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> Hi,
> >>>>> thanks for your replay.
> >>>>>
> >>>>> We are able to export snapshot to another cluster But this is not
> >> what
> >>> we
> >>>>> are looking for.
> >>>>> We want to export snapshot to remote server without copying to local
> >>> file
> >>>>> system.
> >>>>>
> >>>>> Thanks
> >>>>> OC
> >>>>>
> >>>>> On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>>>>
> >>>>>> Here is a sample command exporting to another cluster:
> >>>>>>
> >>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> >>>>>> gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb <[EMAIL PROTECTED]>
> >> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> can someone please let us know if we can export snapshots data
> >>>> directly
> >>>>>> to
> >>>>>>> outside of cluster without copying to local.
> >>>>>>>
> >>>>>>> Please refer us if there are any blog with example.
> >>>>>>>
> >>>>>>> Below command is copying data to local file system under /tmp.
> >>>>>>> hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
> >>>> mytable
> >>>>>>> -copy-to file:///tmp/hbase_backup -mappers 16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB