Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot


+
Ted Yu 2013-04-16, 17:00
+
lars hofhansl 2013-04-16, 17:12
+
Ted Yu 2013-04-16, 21:02
+
Harsh J 2013-04-17, 08:10
+
Azuryy Yu 2013-04-17, 08:19
+
Andrew Purtell 2013-04-17, 16:14
+
Tsz Wo Sze 2013-04-18, 20:50
+
Ted Yu 2013-04-16, 02:13
+
lars hofhansl 2013-04-16, 03:00
+
Ted Yu 2013-04-16, 03:12
+
Ted Yu 2013-04-16, 03:18
+
Azuryy Yu 2013-04-16, 03:23
+
Ted Yu 2013-04-16, 03:26
+
Tsz Wo Sze 2013-04-16, 02:34
+
Azuryy Yu 2013-04-16, 02:33
+
lars hofhansl 2013-04-16, 03:47
+
Ted Yu 2013-04-16, 03:31
+
Enis Söztutar 2013-04-16, 03:52
+
Ted Yu 2013-04-16, 04:00
+
Ted Yu 2013-04-16, 04:17
Copy link to this message
-
Re: collision in the naming of '.snapshot' directory between hdfs snapshot and hbase snapshot
I have logged HBASE-8352

Cheers

On Mon, Apr 15, 2013 at 9:17 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> I have a patch which touched these tests:
>
> http://pastebin.com/P4p8LEAZ
>
> I am running 0.94 test suite now - will publish patch and test result in
> the morning.
>
> Cheers
>
>
> On Mon, Apr 15, 2013 at 9:00 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> bq. Alternatively, we can detect the underlying Hadoop version, and use
>> either .snapshot or .hbase_snapshot in 0.94 depending on h1 & h2.
>>
>> I think this would introduce more confusion, especially for operations.
>>
>> Cheers
>>
>> On Mon, Apr 15, 2013 at 8:52 PM, Enis Söztutar <[EMAIL PROTECTED]>wrote:
>>
>>> Because HDFS exposes the snapshots so that the normal file system
>>> operations are mapped inside snapshot dirs, I think HDFS reserving the
>>> .snapshot name makes sense. OTOH, nothing is specific about the dir name
>>> that is chosen by HBase.
>>>
>>> I would prefer to change the dir name in 0.94 as well, since 0.94 is also
>>> being run on top of hadoop 2. Alternatively, we can detect the underlying
>>> Hadoop version, and use either .snapshot or .hbase_snapshot in 0.94
>>> depending on h1 & h2.
>>>
>>> Enis
>>>
>>>
>>> On Mon, Apr 15, 2013 at 8:31 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>>
>>> > bq. let's make the hbase snapshot for a conf variable.
>>> >
>>> > Once we decide on the new name of snapshot directory, we should still
>>> use
>>> > hardcoded value. This aligns with current code base:
>>> > See this snippet from HConstants:
>>> >
>>> >   public static final List<String> HBASE_NON_TABLE_DIRS >>> >
>>> >     Collections.unmodifiableList(Arrays.asList(new String[] {
>>> > HREGION_LOGDIR_NAME,
>>> >
>>> >       HREGION_OLDLOGDIR_NAME, CORRUPT_DIR_NAME, SPLIT_LOGDIR_NAME,
>>> >
>>> >       HBCK_SIDELINEDIR_NAME, HFILE_ARCHIVE_DIRECTORY,
>>> SNAPSHOT_DIR_NAME,
>>> > HBASE_TEMP_DIRECTORY }));
>>> > Cheers
>>> >
>>> > On Mon, Apr 15, 2013 at 8:24 PM, Jonathan Hsieh <[EMAIL PROTECTED]>
>>> wrote:
>>> >
>>> > > constraints:
>>> > >
>>> > > 1) hbase 0.94.6 is released and .snapshot is hardcoded in there.
>>> > > 2) hdfs snapshots is a Hadoop 2.1 or 3.0 feature. I doubt that it
>>> will
>>> > ever
>>> > > make it to 1.x.  This hdfs feature ideally this shouldn't affect
>>> current
>>> > A
>>> > > pache Hbase 0.94.x's.
>>> > > 3) hbase 95/96 may default to Hadoop1 or Hadoop 2. these versions
>>> should
>>> > > pick a different table snapshot name to respect fs conventions.
>>> > >
>>> > > proposed actions:
>>> > >
>>> > > 1) let's make the hbase snapshot for a conf variable. (hbase.
>>> > > snapshots.dir)  let's change the default for hbase 95+. (maybe
>>> > > .hbase-snapshots). we'll also port this patch to 0.94.x
>>> > > 2) let's publish instructions on how to update the hbase snapshot
>>> dir:
>>> > > shutdown hbase, config update, rename dir, restart hbase.
>>> > > 3) I lean towards leaving the current default hbase snapshot dir in
>>> 94
>>> > > since it shouldn't be affected.  upgrading hbase to 95/96 will
>>> require
>>> > > shutdown and update scripts so it seems like the ideal time to
>>> autoforce
>>> > > this default change.
>>> > >
>>> > > Thoughts?
>>> > >
>>> > >
>>> > > On Monday, April 15, 2013, lars hofhansl wrote:
>>> > >
>>> > > > OK. Let's try to fix that quickly, so that I can release HBase
>>> 0.94.7.
>>> > > >
>>> > > > -- Lars
>>> > > >
>>> > > >
>>> > > >
>>> > > > ________________________________
>>> > > >  From: Ted Yu <[EMAIL PROTECTED]>
>>> > > > To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
>>> > > > Sent: Monday, April 15, 2013 7:13 PM
>>> > > > Subject: collision in the naming of '.snapshot' directory between
>>> hdfs
>>> > > > snapshot and hbase snapshot
>>> > > >
>>> > > >
>>> > > > Hi,
>>> > > > This afternoon Huned ad I discovered an issue while playing with
>>> HBase
>>> > > > Snapshots on top of Hadoop's Snapshot branch (
>>> > > > http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2802/).
>>> > > >
>>
+
Harsh J 2013-04-18, 06:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB