Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot


+
Jean-Marc Spaggiari 2013-04-16, 11:10
+
Jonathan Hsieh 2013-04-16, 03:24
Copy link to this message
-
Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot
I was away from keyboard when I asserted that hdfs snapshot was a hadoop
2.1 or 3.0 feature.  Apparently it is targeted as a hadoop 2.0.5 feature.
 (I'm a little surprised -- expected this to be a hadoop2 compat breaking
feature) -- so I agree that this is a bit more urgent.

Anyway, I agree that the fs .snapshot naming convention is long standing
and should win.

My concern is with breaking compatibility in 0.94 again -- if we don't go
down the conf variable route,  I consider having docs to properly document
how to do the upgrade and caveats of doing the upgrade in the docs/release
notes blocker to hbase 0.94.7.  (specifically mentioning from 0.94.6 to
0.94.7, and to possibly to 0.95).

Jon.

On Mon, Apr 15, 2013 at 9:00 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> bq. Alternatively, we can detect the underlying Hadoop version, and use
> either .snapshot or .hbase_snapshot in 0.94 depending on h1 & h2.
>
> I think this would introduce more confusion, especially for operations.
>
> Cheers
>
> On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:
>
> > Because HDFS exposes the snapshots so that the normal file system
> > operations are mapped inside snapshot dirs, I think HDFS reserving the
> > .snapshot name makes sense. OTOH, nothing is specific about the dir name
> > that is chosen by HBase.
> >
> > I would prefer to change the dir name in 0.94 as well, since 0.94 is also
> > being run on top of hadoop 2. Alternatively, we can detect the underlying
> > Hadoop version, and use either .snapshot or .hbase_snapshot in 0.94
> > depending on h1 & h2.
> >
> > Enis
> >
> >
> > On Mon, Apr 15, 2013 at 8:31 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > bq. let's make the hbase snapshot for a conf variable.
> > >
> > > Once we decide on the new name of snapshot directory, we should still
> use
> > > hardcoded value. This aligns with current code base:
> > > See this snippet from HConstants:
> > >
> > >   public static final List<String> HBASE_NON_TABLE_DIRS > > >
> > >     Collections.unmodifiableList(Arrays.asList(new String[] {
> > > HREGION_LOGDIR_NAME,
> > >
> > >       HREGION_OLDLOGDIR_NAME, CORRUPT_DIR_NAME, SPLIT_LOGDIR_NAME,
> > >
> > >       HBCK_SIDELINEDIR_NAME, HFILE_ARCHIVE_DIRECTORY,
> SNAPSHOT_DIR_NAME,
> > > HBASE_TEMP_DIRECTORY }));
> > > Cheers
> > >
> > > On Mon, Apr 15, 2013 at 8:24 PM, Jonathan Hsieh <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > constraints:
> > > >
> > > > 1) hbase 0.94.6 is released and .snapshot is hardcoded in there.
> > > > 2) hdfs snapshots is a Hadoop 2.1 or 3.0 feature. I doubt that it
> will
> > > ever
> > > > make it to 1.x.  This hdfs feature ideally this shouldn't affect
> > current
> > > A
> > > > pache Hbase 0.94.x's.
> > > > 3) hbase 95/96 may default to Hadoop1 or Hadoop 2. these versions
> > should
> > > > pick a different table snapshot name to respect fs conventions.
> > > >
> > > > proposed actions:
> > > >
> > > > 1) let's make the hbase snapshot for a conf variable. (hbase.
> > > > snapshots.dir)  let's change the default for hbase 95+. (maybe
> > > > .hbase-snapshots). we'll also port this patch to 0.94.x
> > > > 2) let's publish instructions on how to update the hbase snapshot
> dir:
> > > > shutdown hbase, config update, rename dir, restart hbase.
> > > > 3) I lean towards leaving the current default hbase snapshot dir in
> 94
> > > > since it shouldn't be affected.  upgrading hbase to 95/96 will
> require
> > > > shutdown and update scripts so it seems like the ideal time to
> > autoforce
> > > > this default change.
> > > >
> > > > Thoughts?
> > > >
> > > >
> > > > On Monday, April 15, 2013, lars hofhansl wrote:
> > > >
> > > > > OK. Let's try to fix that quickly, so that I can release HBase
> > 0.94.7.
> > > > >
> > > > > -- Lars
> > > > >
> > > > >
> > > > >
> > > > > ________________________________
> > > > >  From: Ted Yu <[EMAIL PROTECTED]>
> > > > > To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> > > > > Sent: Monday, April 15, 2013 7
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]
+
Enis Söztutar 2013-04-17, 17:59