Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot


Copy link to this message
-
Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot
Harsh, the discussion above includes the reasoning behind forcing the
change in hbase rather than hdfs. Although HBase has shipped with this,
HDFS's snapshots are user visible, meaning that you can do:

hadoop fs -ls /user/foo/.snapshot/

Plus, it is a convention for file systems (netapp, etc) to expose snapshots
this way, having a name ".snapshot". HBase's snapshot directories are not
user visible, and not widely used yet.

Enis
On Wed, Apr 17, 2013 at 9:14 AM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> Thanks for the consideration but we've just committed a change to address
> this as HBASE-8352
>
>
> On Wednesday, April 17, 2013, Harsh J wrote:
>
> > Pardon my late inquisition here but since HBase already shipped out
> > with a name .snapshots/, why do we force them to change it, and not
> > rename HDFS' snapshots to use .hdfs-snapshots, given that HDFS
> > Snapshots has not been released for any users yet. The way I see it,
> > that'd be much more easier to do than making a workaround for a done
> > deal on HBase, which already has its snapshot's users.
> >
> > @Tsz-Wo - If the snapshots in HDFS aren't a 'generic' feature
> > applicable to other FileSystem interface implementations as well, then
> > .hdfs-snapshots should be fine for it - no?
> >
> > On Wed, Apr 17, 2013 at 2:32 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > > Hi,
> > > Please take a look at patch v5 attached to HBASE-8352.
> > >
> > > It would be nice to resolve this blocker today so that 0.94.7 RC can be
> > cut.
> > >
> > > Thanks
> > >
> > > On Tue, Apr 16, 2013 at 10:12 AM, lars hofhansl <[EMAIL PROTECTED]>
> > wrote:
> > >
> > >> Please see my last comment on the jira. We can make this work without
> > >> breaking users who are using HDFS snapshots.
> > >>
> > >>   ------------------------------
> > >>  *From:* Ted Yu <[EMAIL PROTECTED]>
> > >> *To:* [EMAIL PROTECTED]
> > >> *Cc:* [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
> > >> *Sent:* Tuesday, April 16, 2013 10:00 AM
> > >> *Subject:* Re: collision in the naming of '.snapshot' directory
> between
> > >> hdfs snapshot and hbase snapshot
> > >>
> > >> Let's get proper release notes for HBASE-8352 .
> > >>
> > >> Either Lars or I can send out notification to user mailing list so
> that
> > >> there is enough preparation for this change.
> > >>
> > >> Cheers
> > >>
> > >> On Tue, Apr 16, 2013 at 8:46 AM, Jonathan Hsieh <[EMAIL PROTECTED]>
> > wrote:
> > >>
> > >> I was away from keyboard when I asserted that hdfs snapshot was a
> hadoop
> > >> 2.1 or 3.0 feature.  Apparently it is targeted as a hadoop 2.0.5
> > feature.
> > >>  (I'm a little surprised -- expected this to be a hadoop2 compat
> > breaking
> > >> feature) -- so I agree that this is a bit more urgent.
> > >>
> > >> Anyway, I agree that the fs .snapshot naming convention is long
> standing
> > >> and should win.
> > >>
> > >> My concern is with breaking compatibility in 0.94 again -- if we don't
> > go
> > >> down the conf variable route,  I consider having docs to properly
> > document
> > >> how to do the upgrade and caveats of doing the upgrade in the
> > docs/release
> > >> notes blocker to hbase 0.94.7.  (specifically mentioning from 0.94.6
> to
> > >> 0.94.7, and to possibly to 0.95).
> > >>
> > >> Jon.
> > >>
> > >> On Mon, Apr 15, 2013 at 9:00 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >>
> > >> > bq. Alternatively, we can detect the underlying Hadoop version, and
> > use
> > >> > either .snapshot or .hbase_snapshot in 0.94 depending on h1 & h2.
> > >> >
> > >> > I think this would introduce more confusion, especially for
> > operations.
> > >> >
> > >> > Cheers
> > >> >
> > >> > On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <[EMAIL PROTECTED]>
> > >> wrote:
> > >> >
> > >> > > Because HDFS exposes the snapshots so that the normal file system
> > >> > > operations are mapped inside snapshot dirs, I think HDFS reserving
> > the
> > >> > > .snapshot name makes sense. OTOH, nothing is specific about the
> dir