Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot


+
Ted Yu 2013-04-16, 17:00
+
lars hofhansl 2013-04-16, 17:12
Copy link to this message
-
Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot
Hi,
Please take a look at patch v5 attached to HBASE-8352.

It would be nice to resolve this blocker today so that 0.94.7 RC can be cut.

Thanks

On Tue, Apr 16, 2013 at 10:12 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Please see my last comment on the jira. We can make this work without
> breaking users who are using HDFS snapshots.
>
>   ------------------------------
>  *From:* Ted Yu <[EMAIL PROTECTED]>
> *To:* [EMAIL PROTECTED]
> *Cc:* [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
> *Sent:* Tuesday, April 16, 2013 10:00 AM
> *Subject:* Re: collision in the naming of '.snapshot' directory between
> hdfs snapshot and hbase snapshot
>
> Let's get proper release notes for HBASE-8352 .
>
> Either Lars or I can send out notification to user mailing list so that
> there is enough preparation for this change.
>
> Cheers
>
> On Tue, Apr 16, 2013 at 8:46 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>
> I was away from keyboard when I asserted that hdfs snapshot was a hadoop
> 2.1 or 3.0 feature.  Apparently it is targeted as a hadoop 2.0.5 feature.
>  (I'm a little surprised -- expected this to be a hadoop2 compat breaking
> feature) -- so I agree that this is a bit more urgent.
>
> Anyway, I agree that the fs .snapshot naming convention is long standing
> and should win.
>
> My concern is with breaking compatibility in 0.94 again -- if we don't go
> down the conf variable route,  I consider having docs to properly document
> how to do the upgrade and caveats of doing the upgrade in the docs/release
> notes blocker to hbase 0.94.7.  (specifically mentioning from 0.94.6 to
> 0.94.7, and to possibly to 0.95).
>
> Jon.
>
> On Mon, Apr 15, 2013 at 9:00 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > bq. Alternatively, we can detect the underlying Hadoop version, and use
> > either .snapshot or .hbase_snapshot in 0.94 depending on h1 & h2.
> >
> > I think this would introduce more confusion, especially for operations.
> >
> > Cheers
> >
> > On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <[EMAIL PROTECTED]>
> wrote:
> >
> > > Because HDFS exposes the snapshots so that the normal file system
> > > operations are mapped inside snapshot dirs, I think HDFS reserving the
> > > .snapshot name makes sense. OTOH, nothing is specific about the dir
> name
> > > that is chosen by HBase.
> > >
> > > I would prefer to change the dir name in 0.94 as well, since 0.94 is
> also
> > > being run on top of hadoop 2. Alternatively, we can detect the
> underlying
> > > Hadoop version, and use either .snapshot or .hbase_snapshot in 0.94
> > > depending on h1 & h2.
> > >
> > > Enis
> > >
> > >
> > > On Mon, Apr 15, 2013 at 8:31 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >
> > > > bq. let's make the hbase snapshot for a conf variable.
> > > >
> > > > Once we decide on the new name of snapshot directory, we should still
> > use
> > > > hardcoded value. This aligns with current code base:
> > > > See this snippet from HConstants:
> > > >
> > > >   public static final List<String> HBASE_NON_TABLE_DIRS > > > >
> > > >     Collections.unmodifiableList(Arrays.asList(new String[] {
> > > > HREGION_LOGDIR_NAME,
> > > >
> > > >       HREGION_OLDLOGDIR_NAME, CORRUPT_DIR_NAME, SPLIT_LOGDIR_NAME,
> > > >
> > > >       HBCK_SIDELINEDIR_NAME, HFILE_ARCHIVE_DIRECTORY,
> > SNAPSHOT_DIR_NAME,
> > > > HBASE_TEMP_DIRECTORY }));
> > > > Cheers
> > > >
> > > > On Mon, Apr 15, 2013 at 8:24 PM, Jonathan Hsieh <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > constraints:
> > > > >
> > > > > 1) hbase 0.94.6 is released and .snapshot is hardcoded in there.
> > > > > 2) hdfs snapshots is a Hadoop 2.1 or 3.0 feature. I doubt that it
> > will
> > > > ever
> > > > > make it to 1.x.  This hdfs feature ideally this shouldn't affect
> > > current
> > > > A
> > > > > pache Hbase 0.94.x's.
> > > > > 3) hbase 95/96 may default to Hadoop1 or Hadoop 2. these versions
> > > should
> > > > > pick a different table snapshot name to respect fs conventions.
> > > > >
> > > > > proposed actions:
+
Harsh J 2013-04-17, 08:10
+
Azuryy Yu 2013-04-17, 08:19
+
Andrew Purtell 2013-04-17, 16:14
+
Tsz Wo Sze 2013-04-18, 20:50
+
Ted Yu 2013-04-16, 02:13
+
lars hofhansl 2013-04-16, 03:00
+
Ted Yu 2013-04-16, 03:12
+
Ted Yu 2013-04-16, 03:18
+
Azuryy Yu 2013-04-16, 03:23
+
Ted Yu 2013-04-16, 03:26
+
Tsz Wo Sze 2013-04-16, 02:34
+
Azuryy Yu 2013-04-16, 02:33
+
lars hofhansl 2013-04-16, 03:47
+
Ted Yu 2013-04-16, 03:31
+
Enis Söztutar 2013-04-16, 03:52
+
Ted Yu 2013-04-16, 04:00
+
Ted Yu 2013-04-16, 04:17
+
Ted Yu 2013-04-16, 06:11
+
Harsh J 2013-04-18, 06:16