Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # dev - Re: svn commit: r1447710 - /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/recovery/RecoverLease.java


Copy link to this message
-
Re: svn commit: r1447710 - /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/recovery/RecoverLease.java
Keith Turner 2013-02-19, 14:44
On Tue, Feb 19, 2013 at 8:25 AM,  <[EMAIL PROTECTED]> wrote:
> Author: ecn
> Date: Tue Feb 19 13:25:42 2013
> New Revision: 1447710
>
> URL: http://svn.apache.org/r1447710
> Log:
> ACCUMULO-1053 wait until a file has been closed before using it for recovery
>
> Modified:
>     accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/recovery/RecoverLease.java
>
> Modified: accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/recovery/RecoverLease.java
> URL: http://svn.apache.org/viewvc/accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/recovery/RecoverLease.java?rev=1447710&r1=1447709&r2=1447710&view=diff
> =============================================================================> --- accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/recovery/RecoverLease.java (original)
> +++ accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/recovery/RecoverLease.java Tue Feb 19 13:25:42 2013
> @@ -20,6 +20,7 @@ import java.io.IOException;
>
>  import org.apache.accumulo.core.Constants;
>  import org.apache.accumulo.core.conf.Property;
> +import org.apache.accumulo.core.util.UtilWaitThread;
>  import org.apache.accumulo.fate.Repo;
>  import org.apache.accumulo.server.master.Master;
>  import org.apache.accumulo.server.master.tableOps.MasterRepo;
> @@ -81,7 +82,10 @@ public class RecoverLease extends Master
>      try {
>        if (fs instanceof DistributedFileSystem) {
>          DistributedFileSystem dfs = (DistributedFileSystem) fs;
> -        dfs.recoverLease(source);
> +        while (!dfs.recoverLease(source)) {
> +          log.info("Waiting for file to be closed " + source.toString());
> +          UtilWaitThread.sleep(1000);
> +        }

Seems like this loop should be moved to isRead() part of repo.  This
has two advantages.  It does not tie up a Fate thread.  Also if lots
of leases need to be recovered it can cycle through all of them
calling recoverLease once, instead of waiting for each one to be done.
Maybe this would spin up a lot of parallel work in the namenode.

>          log.info("Recovered lease on " + source.toString());
>          return new SubmitFileForRecovery(server, file);
>        }
>
>