Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot


Copy link to this message
-
Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot
Pardon my late inquisition here but since HBase already shipped out
with a name .snapshots/, why do we force them to change it, and not
rename HDFS' snapshots to use .hdfs-snapshots, given that HDFS
Snapshots has not been released for any users yet. The way I see it,
that'd be much more easier to do than making a workaround for a done
deal on HBase, which already has its snapshot's users.

@Tsz-Wo - If the snapshots in HDFS aren't a 'generic' feature
applicable to other FileSystem interface implementations as well, then
.hdfs-snapshots should be fine for it - no?

On Wed, Apr 17, 2013 at 2:32 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> Hi,
> Please take a look at patch v5 attached to HBASE-8352.
>
> It would be nice to resolve this blocker today so that 0.94.7 RC can be cut.
>
> Thanks
>
> On Tue, Apr 16, 2013 at 10:12 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
>> Please see my last comment on the jira. We can make this work without
>> breaking users who are using HDFS snapshots.
>>
>>   ------------------------------
>>  *From:* Ted Yu <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]
>> *Cc:* [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
>> *Sent:* Tuesday, April 16, 2013 10:00 AM
>> *Subject:* Re: collision in the naming of '.snapshot' directory between
>> hdfs snapshot and hbase snapshot
>>
>> Let's get proper release notes for HBASE-8352 .
>>
>> Either Lars or I can send out notification to user mailing list so that
>> there is enough preparation for this change.
>>
>> Cheers
>>
>> On Tue, Apr 16, 2013 at 8:46 AM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>>
>> I was away from keyboard when I asserted that hdfs snapshot was a hadoop
>> 2.1 or 3.0 feature.  Apparently it is targeted as a hadoop 2.0.5 feature.
>>  (I'm a little surprised -- expected this to be a hadoop2 compat breaking
>> feature) -- so I agree that this is a bit more urgent.
>>
>> Anyway, I agree that the fs .snapshot naming convention is long standing
>> and should win.
>>
>> My concern is with breaking compatibility in 0.94 again -- if we don't go
>> down the conf variable route,  I consider having docs to properly document
>> how to do the upgrade and caveats of doing the upgrade in the docs/release
>> notes blocker to hbase 0.94.7.  (specifically mentioning from 0.94.6 to
>> 0.94.7, and to possibly to 0.95).
>>
>> Jon.
>>
>> On Mon, Apr 15, 2013 at 9:00 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > bq. Alternatively, we can detect the underlying Hadoop version, and use
>> > either .snapshot or .hbase_snapshot in 0.94 depending on h1 & h2.
>> >
>> > I think this would introduce more confusion, especially for operations.
>> >
>> > Cheers
>> >
>> > On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <[EMAIL PROTECTED]>
>> wrote:
>> >
>> > > Because HDFS exposes the snapshots so that the normal file system
>> > > operations are mapped inside snapshot dirs, I think HDFS reserving the
>> > > .snapshot name makes sense. OTOH, nothing is specific about the dir
>> name
>> > > that is chosen by HBase.
>> > >
>> > > I would prefer to change the dir name in 0.94 as well, since 0.94 is
>> also
>> > > being run on top of hadoop 2. Alternatively, we can detect the
>> underlying
>> > > Hadoop version, and use either .snapshot or .hbase_snapshot in 0.94
>> > > depending on h1 & h2.
>> > >
>> > > Enis
>> > >
>> > >
>> > > On Mon, Apr 15, 2013 at 8:31 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> > >
>> > > > bq. let's make the hbase snapshot for a conf variable.
>> > > >
>> > > > Once we decide on the new name of snapshot directory, we should still
>> > use
>> > > > hardcoded value. This aligns with current code base:
>> > > > See this snippet from HConstants:
>> > > >
>> > > >   public static final List<String> HBASE_NON_TABLE_DIRS >> > > >
>> > > >     Collections.unmodifiableList(Arrays.asList(new String[] {
>> > > > HREGION_LOGDIR_NAME,
>> > > >
>> > > >       HREGION_OLDLOGDIR_NAME, CORRUPT_DIR_NAME, SPLIT_LOGDIR_NAME,
>> > > >
>> > > >       HBCK_SIDELINEDIR_NAME, HFILE_ARCHIVE_DIRECTORY,

Harsh J