Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Re: svn commit: r1239930 - in /hbase/branches/0.89-fb/src: main/java/org/apache/hadoop/hbase/master/SplitLogManager.java test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java


+
Stack 2012-02-03, 04:59
+
Nicolas Spiegelberg 2012-02-03, 18:52
Copy link to this message
-
Re: svn commit: r1239930 - in /hbase/branches/0.89-fb/src: main/java/org/apache/hadoop/hbase/master/SplitLogManager.java test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java
Hi Stack,

Sorry about that. We meant to tag this commit as 89-fb-only, but we are
still figuring out our tagging system. I guess "[master]" would have been
the right tag in this case, since this changes log-splitting functionality.

Thanks,
--Mikhail

On Fri, Feb 3, 2012 at 10:52 AM, Nicolas Spiegelberg <[EMAIL PROTECTED]>wrote:

> @Stack: this is an 89-master change.  It should have that tag.  I'll look
> to figure out why the precommit hook isn't working properly.
>
> On 2/2/12 8:59 PM, "Stack" <[EMAIL PROTECTED]> wrote:
>
> >Mikhail, the below doesn't have the hbase jira in it?
> >St.Ack
> >
> >On Thu, Feb 2, 2012 at 3:30 PM,  <[EMAIL PROTECTED]> wrote:
> >> Author: mbautin
> >> Date: Thu Feb  2 23:30:28 2012
> >> New Revision: 1239930
> >>
> >> URL: http://svn.apache.org/viewvc?rev=1239930&view=rev
> >> Log:
> >> fix fs.delete(path, false) usage
> >>
> >> Summary: Facebook's internal hdfs always fails for fs.delete(path,
> >>false). hdfs
> >> 0.23 works as expected - it will delete path if it is a file or if it
> >>is an
> >> empty directory.  This issue is only applicable to 89-fb, so it does
> >>need to be
> >> ported to HBase trunk.
> >>
> >> Test Plan: modified unit test. the test fails w/o this diff
> >>
> >> Reviewers: kannan, liyintang, pritam
> >>
> >> Reviewed By: pritam
> >>
> >> CC: hbase-eng@lists
> >>
> >> Differential Revision: https://phabricator.fb.com/D400044
> >>
> >>
> >> Modified:
> >>
> >>hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/Split
> >>LogManager.java
> >>
> >>hbase/branches/0.89-fb/src/test/java/org/apache/hadoop/hbase/master/TestS
> >>plitLogManager.java
> >>
> >> Modified:
> >>hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/Split
> >>LogManager.java
> >> URL:
> >>
> http://svn.apache.org/viewvc/hbase/branches/0.89-fb/src/main/java/org/apa
> >>che/hadoop/hbase/master/SplitLogManager.java?rev=1239930&r1=1239929&r2=12
> >>39930&view=diff
> >>
> >>========================================================================> >>====> >> ---
> >>hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/Split
> >>LogManager.java (original)
> >> +++
> >>hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/master/Split
> >>LogManager.java Thu Feb  2 23:30:28 2012
> >> @@ -275,17 +275,19 @@ public class SplitLogManager implements
> >>       for (Path logDir : logDirs) {
> >>         status.setStatus("Cleaning up log directory...");
> >>         try {
> >> -          if (fs.exists(logDir) && !fs.delete(logDir, false)) {
> >> -            LOG.warn("Unable to delete log src dir. Ignoring. " +
> >>logDir);
> >> +          if (fs.exists(logDir)) {
> >> +            FileStatus[] files = fs.listStatus(logDir);
> >> +            if (files == null || files.length == 0) {
> >> +              if (fs.delete(logDir, true) == false) {
> >> +                LOG.warn("Unable to delete log src dir. Ignoring. " +
> >>logDir);
> >> +              }
> >> +            } else {
> >> +              LOG.warn("returning success without actually splitting
> >>and " +
> >> +                  "deleting all the log files in path " + logDir);
> >> +            }
> >>           }
> >>         } catch (IOException ioe) {
> >> -          FileStatus[] files = fs.listStatus(logDir);
> >> -          if (files != null && files.length > 0) {
> >> -            LOG.warn("returning success without actually splitting and
> >>" +
> >> -                "deleting all the log files in path " + logDir);
> >> -          } else {
> >> -            LOG.warn("Unable to delete log src dir. Ignoring. " +
> >>logDir, ioe);
> >> -          }
> >> +          LOG.warn("Unable to delete log src dir. Ignoring. " +
> >>logDir, ioe);
> >>         }
> >>       }
> >>       tot_mgr_log_split_batch_success.incrementAndGet();
> >>
> >> Modified:
> >>hbase/branches/0.89-fb/src/test/java/org/apache/hadoop/hbase/master/TestS
> >>plitLogManager.java
> >> URL:
> >>
> http://svn.apache.org/viewvc/hbase/branches/0.89-fb/src/test/java/org/apa
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB