Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # dev - Review Request: Fix Sqoop unit test failures on Windows


+
Ahmed El Baz 2013-04-01, 20:48
+
Venkat Ranganathan 2013-04-05, 16:21
+
Venkat Ranganathan 2013-04-09, 15:02
+
Ahmed El Baz 2013-04-09, 00:33
+
Venkat Ranganathan 2013-04-08, 23:30
+
Ahmed El Baz 2013-04-08, 22:28
+
Ahmed El Baz 2013-04-12, 02:36
Copy link to this message
-
Re: Review Request: Fix Sqoop unit test failures on Windows
Venkat Ranganathan 2013-04-10, 05:30


> On April 5, 2013, 4:21 p.m., Venkat Ranganathan wrote:
> > src/java/org/apache/sqoop/util/ClassLoaderStack.java, line 78
> > <https://reviews.apache.org/r/10229/diff/1/?file=276974#file276974line78>
> >
> >     Why is the JarURL format getting changed.   File(jarFile).toURI().toURL() will still have the scheme as file right?
>
> Ahmed El Baz wrote:
>     The objective is to construct the URL path using URL class which resolves the failures on Windows due to the preceding "jar:" and ending "!/" parts. I believe this is also favorable than handcrafting the URL string.
>    
>     Verified this is working on both Windows and Linux
>
> Venkat Ranganathan wrote:
>     Are you saying this does not work on Windows?
>    
>     String urlPath = "jar:file:" + new File(jarFile).getAbsolutePath() + "!/";
>
> Ahmed El Baz wrote:
>     Correct. Using the crafted String urlPath is failing on Windows, thus the need to use the URL class.
>
> Venkat Ranganathan wrote:
>     That is interesting. remember using it in my earlier work.  I understand that you want to get it consistent across Unix and windows.   Can you please see the content below and try it on Windows and Linux? I don't think the issue is because of jar: and !/ suffixes.   The issue is because file urls crafted with file:// syntax expect the file part to start with / and not a drive letter.   That is why using file: (without // after file) helps.  In any case, URLClassLoader can work with Jar URLs without the prefix if we want the full jarfile contents.   I think there is a windows issue with dynamically replacing jarfiles because of a Java bug.   That is a different issue.
>    
>     ===========>     import java.io.File;
>     import java.net.JarURLConnection;
>     import java.net.URL;
>     import java.util.Map;
>     import java.util.jar.Attributes;
>     import java.util.jar.Manifest;
>    
>    
>    
>     public class JarURLConn {
>    
>     public static void main(String args[]) {
>     if (args.length < 1) {
>     System.out.println("Usage JavaURLConn jar1 ...");
>     System.exit(1);
>     }
>     for (int i = 0; i < args.length; ++i) {
>     System.out.println("Information on jar file " + args[i]);
>     System.out.println("\n\n");
>     try {
>     printJarInfo(args[i]);
>     }
>     catch (Exception e) {
>     System.out.println("Caught exception : " + e.toString());
>     }
>     }
>     }
>    
>    
>     private static void printJarInfo(String jarFile) throws Exception {
>     File f = new File(jarFile);
>     if (f.canRead()) {
>     StringBuilder sb = new StringBuilder();
>     sb.append("jar:file:");
>     sb.append(f.getAbsolutePath());
>     sb.append("!/");
>     String jarURLStr = sb.toString();
>                             System.out.println("Jar URL is " + jarURLStr);
>     URL url = new URL(jarURLStr);
>     JarURLConnection conn = (JarURLConnection) url.openConnection();
>     Manifest manifest = conn.getManifest();
>     Map<String, Attributes> attrMap = manifest.getEntries();
>    
>     for (String key : attrMap.keySet()) {
>     Attributes attrs = attrMap.get(key);
>     for (Object attrKey : attrs.keySet()) {
>     System.out.println("Attribute " + attrKey + " = "
>     + attrs.get(attrKey));
>     }
>     }
>    
>     }
>     else {
>     System.out.println("Can't access jar file " + jarFile);
>     }
>    
>     }
>     }
>     ======>
> Ahmed El Baz wrote:
>     Thanks Venkat,
>     I have tried the sample above, and yes the error happens when we add the extra "://". Are you suggesting to change the fix to remove the slashes from the file:// part. I believe using URL APIs would abstract all those issues. Please let me know what you think

I think to be safe (and correct in terms of usage and convention), please leave the scheme as jar - so the constructed URL (with your change for platform neutral way of creating a file URL) can be
    "jar:" + new File(jarFile).toURI().toURL().toString() + "!/"
- Venkat
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10229/#review18705
On April 1, 2013, 8:48 p.m., Ahmed El Baz wrote:
+
Ahmed El Baz 2013-04-09, 21:38
+
Ahmed El Baz 2013-04-12, 02:31
+
Venkat Ranganathan 2013-04-14, 04:57
+
Jarek Cecho 2013-04-14, 06:24
+
Ahmed El Baz 2013-04-15, 16:47