Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot


+
Ted Yu 2013-04-16, 17:00
+
lars hofhansl 2013-04-16, 17:12
+
Ted Yu 2013-04-16, 21:02
+
Harsh J 2013-04-17, 08:10
+
Azuryy Yu 2013-04-17, 08:19
+
Andrew Purtell 2013-04-17, 16:14
+
Tsz Wo Sze 2013-04-18, 20:50
+
Ted Yu 2013-04-16, 02:13
+
lars hofhansl 2013-04-16, 03:00
+
Ted Yu 2013-04-16, 03:12
+
Ted Yu 2013-04-16, 03:18
+
Azuryy Yu 2013-04-16, 03:23
+
Ted Yu 2013-04-16, 03:26
+
Tsz Wo Sze 2013-04-16, 02:34
+
Azuryy Yu 2013-04-16, 02:33
+
lars hofhansl 2013-04-16, 03:47
+
Ted Yu 2013-04-16, 03:31
+
Enis Söztutar 2013-04-16, 03:52
Copy link to this message
-
Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot
bq. Alternatively, we can detect the underlying Hadoop version, and use
either .snapshot or .hbase_snapshot in 0.94 depending on h1 & h2.

I think this would introduce more confusion, especially for operations.

Cheers

On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:

> Because HDFS exposes the snapshots so that the normal file system
> operations are mapped inside snapshot dirs, I think HDFS reserving the
> .snapshot name makes sense. OTOH, nothing is specific about the dir name
> that is chosen by HBase.
>
> I would prefer to change the dir name in 0.94 as well, since 0.94 is also
> being run on top of hadoop 2. Alternatively, we can detect the underlying
> Hadoop version, and use either .snapshot or .hbase_snapshot in 0.94
> depending on h1 & h2.
>
> Enis
>
>
> On Mon, Apr 15, 2013 at 8:31 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > bq. let's make the hbase snapshot for a conf variable.
> >
> > Once we decide on the new name of snapshot directory, we should still use
> > hardcoded value. This aligns with current code base:
> > See this snippet from HConstants:
> >
> >   public static final List<String> HBASE_NON_TABLE_DIRS > >
> >     Collections.unmodifiableList(Arrays.asList(new String[] {
> > HREGION_LOGDIR_NAME,
> >
> >       HREGION_OLDLOGDIR_NAME, CORRUPT_DIR_NAME, SPLIT_LOGDIR_NAME,
> >
> >       HBCK_SIDELINEDIR_NAME, HFILE_ARCHIVE_DIRECTORY, SNAPSHOT_DIR_NAME,
> > HBASE_TEMP_DIRECTORY }));
> > Cheers
> >
> > On Mon, Apr 15, 2013 at 8:24 PM, Jonathan Hsieh <[EMAIL PROTECTED]>
> wrote:
> >
> > > constraints:
> > >
> > > 1) hbase 0.94.6 is released and .snapshot is hardcoded in there.
> > > 2) hdfs snapshots is a Hadoop 2.1 or 3.0 feature. I doubt that it will
> > ever
> > > make it to 1.x.  This hdfs feature ideally this shouldn't affect
> current
> > A
> > > pache Hbase 0.94.x's.
> > > 3) hbase 95/96 may default to Hadoop1 or Hadoop 2. these versions
> should
> > > pick a different table snapshot name to respect fs conventions.
> > >
> > > proposed actions:
> > >
> > > 1) let's make the hbase snapshot for a conf variable. (hbase.
> > > snapshots.dir)  let's change the default for hbase 95+. (maybe
> > > .hbase-snapshots). we'll also port this patch to 0.94.x
> > > 2) let's publish instructions on how to update the hbase snapshot dir:
> > > shutdown hbase, config update, rename dir, restart hbase.
> > > 3) I lean towards leaving the current default hbase snapshot dir in 94
> > > since it shouldn't be affected.  upgrading hbase to 95/96 will require
> > > shutdown and update scripts so it seems like the ideal time to
> autoforce
> > > this default change.
> > >
> > > Thoughts?
> > >
> > >
> > > On Monday, April 15, 2013, lars hofhansl wrote:
> > >
> > > > OK. Let's try to fix that quickly, so that I can release HBase
> 0.94.7.
> > > >
> > > > -- Lars
> > > >
> > > >
> > > >
> > > > ________________________________
> > > >  From: Ted Yu <[EMAIL PROTECTED]>
> > > > To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> > > > Sent: Monday, April 15, 2013 7:13 PM
> > > > Subject: collision in the naming of '.snapshot' directory between
> hdfs
> > > > snapshot and hbase snapshot
> > > >
> > > >
> > > > Hi,
> > > > This afternoon Huned ad I discovered an issue while playing with
> HBase
> > > > Snapshots on top of Hadoop's Snapshot branch (
> > > > http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2802/).
> > > >
> > > > HDFS (built from HDFS-2802 branch) doesn't allow paths with .snapshot
> > as
> > > a
> > > > component while HBase tries to create paths with .snapshot as a
> > > component.
> > > > This leads to issues in HBase, and one of HDFS or HBase needs to give
> > up
> > > > the .snapshot reserved keyword. HBase released Snapshots feature in
> > > 0.94.6
> > > > (quite recently) and it may not be too late to change HBase to use a
> > > > different path component in an upcoming new release.
> > > >
> > > > In HBase these path names are not user visible. If there is a
> > deployment
+
Ted Yu 2013-04-16, 04:17
+
Ted Yu 2013-04-16, 06:11
+
Harsh J 2013-04-18, 06:16